Reinforcement Learning and Universal Agent

Abstract

With the recent advancements and developments of deep learning techniques, 'reinforcement learning,' a framework based on the interaction between an agent and the environment, attracts a great deal of attention. This presentation introduces a universal agent model called AIXI (AIξ) proposed by Marcus Hutter (references 1,2). The AIXI model is based on the algorithmic information theory founded by Ray Solomonoff and uses a universal prior distribution in the agent optimization strategy. Based on this formulation, the AIXI can be interpreted as an agent model that can take an optimal strategy under any circumstances. The formulation of such universal agent is an attempt to answer the fundamental question of universal intelligence and may give some hints on how to deepening the reinforcement learning.