Abstract
Overall, the structural simplicity of the network has a strong relationship to its ability to promote generalization. With regards to this type of situation. Rumelhart proposed a new learning algorithm which has a common algorithm for backpropagation (BP) learning and an additional penalizing term, relating to the network complexity, which, when working in tandem, increase the ability of the back propagation learning procedure. This learning algorithm has a property that modifies the weighting factors, thereby, decreasing the total sum of both the errors and the complexity of the network simultaneously.
The authors provide insight as to the implications of the above modified BP algorithm and furnish several simulations which test its performance. Moreover, a refining procedure is provided, this is based on the investigations into the balance between the error term and the complexity term. Finally, the refined algorithm is successfully applied to the systems identification problem and subsequently, shows the desired performance for generalization in neural networks.
