Abstract
It is well known that Backpropagation algorithm is one of the most basic algorithms in neural networks(NNs). Its role can not be overestimated in the field of neural networks. In this paper we propose a new variant of backpropagation algorithm through controlling the outputs of the hidden layers. The proposed algorithm therefore provides better generalization results than the basic backpropagation algorithm. The added term to the criterion function has the following property: (1) Small added noises in the inputs to the networks will not give evident effects to the outputs of the networks; (2) Small added noises in the weight matrix except the one between the hidden layer and output layer will not give large effects to the outputs of the networks. In addition, simulation comparisons are also made between the new algorithm and some conventional regularization methods, such as Laplace and Gaussian regularizer. Simulation results on the two-spiral problem, function approximation problem and Iris data classification problem confirm this assertion.
