Proposal Neural Network with Stochastic Connection and Error Propagation based on Free Energy Principle

Abstract

This paper proposes a neural network model that consists of stochastic connections among neurons, and also proposes a new gradient estimator based on free energy principle for the proposed model. Common neural networks consist of forward propagation and backward propagation intrinsically. Both those propagations need synchronous computation through input to output, which have the problem that they lack locality in computation when the model is defined deeply. As a result, there are 2 problems that the model does not realize a property human brain have and that the requirment to construct appropriate processing hardware is too strict. The proposed model can solve those problems by diveding the propagations in local area and distributing energy to both direction of propagations consistently. The paper also shows neumerical experiments to support the proposal.