Sparse Modeling and Statistical Mechanical Informatics

Abstract

Recent technical advance in statistical mechanical informatics and its application to sparse modeling are explained. The focus is on a systematic derivation of a meanfield-type inference algorithm based on the belief-propagation from Bayesian inference and the cavity method from statistical mechanics. The derived algorithm is called approximate message passing, and its macroscopic behavior is analyzed by the so-called state evolution and has a direct connection to the equations of state derived using the replica method in the replica symmetric level. This framework is applied to compressed sensing in sparse modeling, which is a signal-processing technique exploiting the inherent sparseness of the true signal expressed in an appropriate basis. The limits of perfect reconstruction of the signal are derived using these techniques in the Bayesian and regularized linear regression frameworks, clarifying their natures as dynamical phenomena and phase transitions.