Abstract
The multi-layered neural networks with back-propagation learning algorithms are widely used in pattern recognition systems. However, when these networks are applied to a pattern recognition process, some problems arise in the network behavior and the network structure itself. Those problems include the enormous learning time, the convergence to local minima and the indistinct criterion for the better network structure. Here, a hybrid learning algorithm is proposed for organizing the structure of the multi-layered neural networks. The proposed pruning algorithm consists of two already known algorithms, the structural learning algorithm with forgetting and the optimal brain damage algorithm using the second derivatives of the assessment. After the network is slimmed by the structural learning algorithm with forgetting, unimportant weights are removed from the network using the second derivatives. The simulations are performed for the operation of the Boolean function and the acoustic diagnosis of compressors. The results show that the proposed algorithm is effective for eliminating the unimportant weights. The relation between the second derivatives and the absolute values of weights is discussed.
