An ANN Learning Algorithm Based on Hierarchical Clustering of Training Data

Abstract

We propose a new ANN learning algorithm based on hierarchical clustering of training data. The proposed algorithm first constructs a tree of sub-learning problems by hiearchically clustering given learning patterns in a bottom-up manner and decides a corresponding network structure. The proposed algorithm trains the whole network giving teacher signals of the original learning problem to the output units, and trains sub-networks giving teacher signals of the divided sub-learning problems to the hidden units simultaneously. The hidden units which learn sub-learning problems become feature detectors, which promote the learning of the original learning problem. We demonstrate the advantages of our learning algorithm by solving N-bits parity problems, a non-liner function approximation, iris classification problem, and two-spirals problem. Experimen-tal results show that our learning algorithm obtains better solutions than the standard back-propagation algorithms and one of constructive algorithms in terms of the learning speed and the convergence rate.