A Reinforcement Learning with Evolutionary State Recruitment Strategy for Autonomous Mobile Robots Control

Abstract

In recent robotics fields, much attention has been focused on utilizing reinforcement learning for designing robot controllers, since environments the robots will be situated could be unpredictable for human designers in advance. However there exist some difficulties, one of them is well known as “curse of dimensionality problem” in which as a state space for a learning system (e.g. robot) becomes continuous and high dimensional, the learning process results in time-consuming. Therefore, so as to adopt reinforcement learning for such complicated systems, not only adaptability but also computational efficiencies should be taken into account. In this paper, an evolutionary state recruitment strategy for an actor-critic reinforcement learning system based on NGnet is proposed, which enables the learning system to divide/rearrange its state space gradually according to the task complexity and the progress of learning. Simulation results and real robot implementations of a peg pushing robot control task show the validity of the method.