Effective sampling on Gaussian-Bernoulli restricted Boltzmann machines

Abstract

Gaussian-Bernoulli restricted Boltzmann machine (GBRBM) is a bipartite-type Markov random field consisting of two types of layers, namely, visible and hidden layers, and it can treat continuous data points. For the implementation of GBRBM (e,g, for learning and inference), the evaluation of the expectations of variables is critical; however, this is difficult because of the problem of combinatorial explosion. In this study, we propose an effective method for expectation evaluation on a (canonicalized) GBRBM based on spatial Monte Carlo integration and marginalized-space Gibbs sampling (mGS). Here, mGS is a collapsed-Gibbs-sampling version of the layer-wise blocked Gibbs sampling method and it can reduce the relaxation times of Gibbs sampling. The results of numerical experiments demonstrate the effectiveness of the proposed method.