階層型ニューラルネットワークの情報量基準

抄録

In engineering fields, one of the most important application of artificial neural networks is a non-linear parametric model that can approximate any continuous input-output relation. The quality of approximation depends on the architecture of the neural network used, as well as on the complexity of the target relation. Usually, we do not have accurate input-output relation and we can utilize only input-output examples. In such a case, if a fixed architecture of networks is given, the problem of finding a suitable set of parameters that approximate an unknown relation is usually solved using supervised learning algorithms. Supervised learning is carried out based on a training set which consists of a number of examples observed from an actual system. An important but difficult problem is to determine the optimal number of parameters. This paper presents a statistical approach to this problem of model selection, or determination of the number of hidden units. In other words, we describe a measure of neural networks reliability statistically. The relation between the training error, which is reduced by training process, and the generalization error, which is a measure of the quality of approximation, is derived by the following two gaps: ・One is a statistical gap between the optimal and the semi-optimal solution caused by the finite training examples. ・The other is a gap between the optimal and estimated solution caused by the learning process. In this paper, we describe that the gap between the optimal and the semi-optimal solution is measured by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful models and the gap between the semi-optimal and estimated solution is estimated by a cross validation. The above gaps can explain the relation between the training error and the generalization error. This relation leads to a Neural Network Information Criterion (NNIC) which is useful for selecting the optimal network model based on a given training set. We present that this criterion is useful with a simple simulation.