Lyapunov exponent analysis for multilayer neural networks

Abstract

A multilayer neural network can be regarded as a type of discrete-time dynamical system in the sense that layer-to-layer information propagation can be expressed as the time evolution of a particular dynamical system. In this study, we investigate the stability of information propagation in multilayer neural networks for classification problems using finite-time maximum Lyapunov exponents, and we discuss how multilayer neural networks classify inputs. The dynamical stability analysis in this study reveals the input-dependent stability of trained multilayer neural networks. Multilayer neural networks are trained such that the information propagation is highly sensitive to input data vectors near a decision boundary for classification whereas it is less sensitive to input data vectors far from the decision boundary. This implies that the decision boundary in classification problems is characterized by a set where the finite-time maximum Lyapunov exponents of the information propagation are relatively large. These results offer a new perspective on the estimation of uncertainty of classification using multilayer neural networks.