Abstract
In this paper, on classification problems, dependency of generalization capability for multi-layered neural networks on their number of hidden units is discussed. First, a simple model representing the relationship between generalization capability and the number of hidden units is established, which has two characteristic behaviors, namely,
(1) generalization capability increases with the number of hidden units, then saturates; and
(2) the saturation number of hidden units increases with noise level of the generalized data. According to this model, the saturation number of hidden units can be defined to be optimal, which is the smallest and sufficient for the maximum generalization capability. Then a statistical algorithm based on linear regression analysis proposed in the previous paper is extended to accurately determine the optimal number of hidden units.
By the simulation using two typical neural network classification systems, i.e. alphabet recognition and practical number recognition systems, the developed model is found to describe the above relationship properly. Also the extended algorithm is demonstrated to be effective to estimate the optimal number of hidden units.
