Abstract
This study presents a new method to prune hidden units in trained recurrent neural networks. Our approach is based on Moore's method for reduction of linear systems. To improve generalization ability and to reduce computational complexity, pruning methods of neural networks have been actively studied. However the choice of pruned neurons has been more or less empirical in previous methods. In the present method we first linearize the output function of neurons to approximate trained recurrent networks by linear systems. Then Moore's method, which removes the less controllable/observable subsystem after the original system is coordinately transformed to an internally balanced system, is applied to the resultant linear system for size reduction. Finally the reduced system is converted to a non-linear recurrent network for retraining. Numerical results show that our method can prune hidden units successfully with a small amount of retraining computation.
