Real-World Robot Control by Deep Active Inference with Temporally Hierarchical World Model

Abstract

Deep learning-based robots are expected to achieve various goals in real-world environments. To realize this, it is essential to handle environmental uncertainties using both goal-directed and exploratory actions. Deep active inference (DAIf) offers a promising approach but suffers from high computational costs and requires strong representational capabilities for modeling environmental dynamics. To address these challenges, we propose a novel DAIf framework. The framework comprises a hierarchical world model, an abstract world model, and an action model. The hierarchical world model learns environmental dynamics by introducing a temporal hierarchy, enhancing its representational capability. The action model learns latent states of action sequences as abstract actions. The abstract world model learns the relationship between the hierarchical world model's representation and the abstract actions, reducing computational costs. Robotic object manipulation experiments in uncertain environments demonstrated that the framework reduced computational costs compared to conventional approaches, achieved diverse goals, and generated exploratory actions to address environmental uncertainties.