Progressive Data Increasing as Initialization of Neural Network

Abstract

Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of tasks. With the improvement of the hardware performance, the structure of the neural network becomes more and more complicated, and the amounts of data used for training networks are becoming larger.Although these changes greatly contributed to the recognition accuracy of the neural network, it often leads to training instability in some cases.In this paper, we propose to use a subset of training data for initialization of neural networks. We found that it can help us to stabilize the training process.