Abstract
When we apply a hierarchical neural network based on the back-propagation algorithm to a particular problem, we must determine beforehand the suitable size of network for the problem. But it is a very difficult problem. Too small a network will not learn at all, while too large a network will be inefficient and worsen its generalization ability due to overfitting.
In order to solve this problem, in this paper we propose a compact structuring method based on learning with a large size network and then compacting gradually the suitable size network by combining extra hidden units. The result is a small and efficient network that performs better than the original. Also we demonstrate the effectiveness of this method by appling it to two problems, i. e., to identify a logic function and to recognize handwriting characters.
