Learning of Agent Behavior Based on Hierarchical Modular Reinforcement Learning

Abstract

Reinforcement learning is a promising approach to realize intelligent agent such as autonomous mobile robots. In order to apply the reinforcement learning to actual sized problem, the ""curse of dimensionality"" problem in partition of sensory states should be avoided maintaining computational efficiency. The paper describes a hierarchical modular reinforcement learning that memory based modular Profit Sharing learning algorithm is combined with Q Learning reinforcement learning algorithm hierarchically in multi-agent environment. By using the hierarchical modular reinforcement learning, this research aims at learning the behavior of the agent in the pursuit problem, i.e. reinforcement learning problem in the multi-agent environment. Through numerical experiments, we found that the proposed method has good convergence property of learning compared with the conventional algorithms.