Modifying Desired Outputs to Improve Pattern Recognition by Combining Subfeature-Input Neural Networks

Abstract

We have investigated ways to improve pattern recognition ability by combining several small back-propagation neural networks (BPNNs) into a modular-net architecture. In this architecture several subfeatures are extracted from patterns, each subfeature is input into a separate BPNN, and the output vectors from the BPNNs are combined to obtain the recognition results. Using two subfeatures extracted from handwritten digits, we investigated how best to obtain the desired outputs for similar patterns in order to improve the generalization ability of the modular-net architecture. We found that the conventional all-or-nothing desired-output approach, “1” for the correct class and “0” for the other classes, prevents the BPNN outputs for similar classes from becoming sufficiently large, preventing the combined output for the correct class for patterns in which both subfeatures are very similar to those of the other classes from becoming maximal. We also found that modifying the desired outputs according to the similarity of the input patterns (i.e., increasing desired outputs to similar classes) increases the BPNN outputs for similar classes, which help maximize the combined output for the correct class, thus improving the generalization ability of the modular-net architecture. The effectiveness of our approach was shown by several experiments using two subfeatures extracted from handwritten digits.