Analysis on equilibrium point of expectation propagation using information geometry

Abstract

Expectation propagation (EP) extends belief propagation in that EP allows users to arbitrarily choose statistics that are used in its approximation scheme. In this paper, we discuss the problem of how a choice of statistics affects performance of EP, using information geometry. Second-order perturbation is used to analyze estimation error of EP. We compare estimation errors of two instances of EP formulation, with one instance including, and therefore being an extension to, the other. Our analytical and numerical results show that choosing more statistics for inclusion in the EP formulation does not necessarily yield smaller estimation errors.