A Study on Construction of Deep Neural Networks Based on Autoencoders for Each Category

Abstract

In recent years, Deep Neural Network (DNN) has attracted attention. In the back propagation method, one of the learning methods in DNN, the initial value of the weight in the neuron greatly affects the accuracy of the prediction result. Therefore, it is necessary to set an appropriate initial value when learning. One way to give the initial value is to obtain the initial value by pre-training. In this study, we focus on this pre-training and propose an initial value setting that reflects the feature value of the category using Autoencoder (AE). As a result, the initial value having the characteristic for each category is assigned to each branch as a weight, which is thought to contribute to the improvement of the classification accuracy. In this study, we evaluate the effectiveness of this method by using Yomiuri Shimbun article data.