Reinforcement Learning with Autonomous Segmentation for Continuous State and Action Spaces

Abstract

One of major research topics for behavior-based AI is to construct an appropriate sensor-motor, relation for an autonomous moving robot in an embedded environment, hopefully, with less preliminary setting by an autonomous robot designer. This paper proposes a new reinforcement learning algorithm, which is called the Continuous Space Classifier Generator (CSCG), for this problem. The major attraction of CSCG is that the state space and the action space of a learning agent are segmented simultaneously with the process of its behavior rule acquisition in the embedded environment. This means that a robot designer can be released from the segmentation of those two spaces, which is often crucial to the success of reinforcement learning. After showing the detail of CSCG, not only computer simulations but also experiments using a small real robot are conducted in order to illustrate the learning process of the proposed method.