A Method for Obtaining the Suitable Sized Neural Network Structure by Multiplication and Combination of Hidden Units

Abstract

When we apply a multi-layered neural network based on the back-propagation algorithm to a particular problem, we must determine beforehand the suitable sized network for the problem. But it is a very difficult problem. Too small a network will not learn at all, while too large a network will be inefficient and worsen its generalization ability due to overfitting the training data.
In order to solve this problem, in this paper we propose a structuring method for obtaining the suitable sized network by multipling and/or combining hidden units in the multi-layered neural network. The result is a compact and efficient network that performs better than the original. Also we demonstrate the effectiveness of this method by appling it to two problems, i.e., to identify a logic function, and to divide a plane.