Exact solvable deep Boltzmann machine

Abstract

In training and inference in a deep Boltzmann machine (DBM), the evaluation of the free energy is crucial. Due to combinatorial explosion, its evaluation is computationally infeasible. We investigate the free energy evaluation of the DBM based on the Bethe free energy representation, which is related to the belief propagation algorithm. This representation significantly reduces the computational cost of the evaluation and enables the exact computation of the free energy in a slim DBM, i.e., a DBM with small-sized layers. Finally, we discuss the properties of the free energy in slim DBMs based on numerical and analytical methods.