IEEJ Transactions on Industry Applications
Online ISSN : 1348-8163
Print ISSN : 0913-6339
ISSN-L : 0913-6339
Dependency of Generalization Capability for a Multi-Layered Neural Network on its Number of Hidden Units
Masahiro KayamaShigeo AbeHiroshi TakenagaYasuo Morooka
Author information

1993 Volume 113 Issue 3 Pages 341-348


In this paper, on classification problems, dependency of generalization capability for multi-layered neural networks on their number of hidden units is discussed. First, a simple model representing the relationship between generalization capability and the number of hidden units is established, which has two characteristic behaviors, namely,
(1) generalization capability increases with the number of hidden units, then saturates; and
(2) the saturation number of hidden units increases with noise level of the generalized data. According to this model, the saturation number of hidden units can be defined to be optimal, which is the smallest and sufficient for the maximum generalization capability. Then a statistical algorithm based on linear regression analysis proposed in the previous paper is extended to accurately determine the optimal number of hidden units.
By the simulation using two typical neural network classification systems, i.e. alphabet recognition and practical number recognition systems, the developed model is found to describe the above relationship properly. Also the extended algorithm is demonstrated to be effective to estimate the optimal number of hidden units.

Information related to the author
© The Institute of Electrical Engineers of Japan
Previous article Next article