A Study for Keeping Generalization Ability of Multilayered Neural Networks using Evolution Strategies

Abstract

This paper proposes a new algorithm for advancing the generalization ability of multilayer neural networks. The proposed algorithm, based on regularization theory, is a method for determining the regularization parameter, on condition that the training data is shown additionally. It is not a method that solves a problem for all training data again when additional training data is shown, but rather a method that adjusts the regularization parameter to fit additional training data. The characteristics of this algorithm are (1) the prediction error for the additional data is used in evaluating to determine the regularization parameter, (2) Evolution strategies (ES) that is multipoint search method is used for the determination problem of regularization parameter. The evaluation of the regularization parameter varies according to the data added. This study simulated an additional learning problem to examine the performance of the proposed method. And the simulation results are presented in this paper.