Deep Active Inference with Reconstructive and Contrastive Learning

Abstract

Tackling perception and action of artificial agents by leveraging deep learning has gained significant attention from the research community. Deep active inference is a particularly promising approach integrating deep learning with the free energy principle: a framework describing cognitive function. In deep active inference, agents learn to minimize the (variational) free energy, an upper bound on Shannon surprise. The recently proposed contrastive free energy uses contrastive methods in place of likelihood computations to reduce complexity. However, the relationship between free energy and contrastive free energy has not been clearly demonstrated. We propose a novel upper bound of surprise that incorporates both reconstructive and contrastive learning based on a single parameter, α, and show that both kinds of free energy can be derived from it. We train agents by minimizing the proposed surprise upper bound under multiple values of α in two simulated environments. Our results suggest that the agent's attention changes depending on α and that combining both reconstructive and contrastive learning provides better performance than using either on its own.