A Constructive Learning Algorithm Based on Division of Training Data for Multilayer Neural Networks

Abstract

It is difficult to decide the optimal neural network structure for a particular problem. We propose a solution to this problem, a new constructive learning algorithm based on division of a given learning problem. The proposed method first decomposes the original learning problem into small pieces and constructs a set of small networks which independently learn one of decomposed problems. It constructs a large network which learns the given learning problem by combining the small networks in a bottom-up manner. We demonstrate the efficiency of our learning algorithm by applying it to XOR, 3-bits parity, a non-liner function approximation, and two-spirals problem. Experimental results show that our learning algorithm can construct networks which have higher learning convergence rate and better generalization capability within less computation time than the standard back-propagation algorithms.