Policy Selection by a Real-World Robot Based on Preference Precision in Deep Active Inference

Abstract

Realizing robots capable of performing various tasks in diverse environments remains an important research goal.In this study, we incorporated a mechanism into a robot that enables it to exhibit both exploratory and goal-directed behaviors. We applied deep learning to active inference, in which the agent is more likely to choose policies that minimize the expected free energy.The expected free energy includes terms that encourage both exploratory and goal-directed behaviors, and the balance between these is determined by the precision of preferences representing the goal. Consequently, active inference agents are expected to select both exploratory and goal-directed behaviors appropriately by adjusting preference precision. Our approach uses a world model to predict future sensory and hidden states. From these predicted states, the expected free energy is calculated. We also limited the number of policies considered by the robot.The experimental results showed that the robot switched between exploratory behavior and goal-directed behavior through the adjustment of preference precision.