Terrain Parameterization in Curriculum Reinforcement Learning for Legged Locomotion Robots

Abstract

In robot control using reinforcement learning, it is becoming common to acquire polices in a simulation environment and then apply them to a real environment. Since there is a gap between their environments, several methods have been proposed for bridging the gap by training robots in various simulation environments. In this work, we propose a curriculum reinforcement learning method for robots that can walk in various terrains. For the curriculum learning, the terrain in the simulation environment is represented by an Ising model and its interaction parameter is used to determine the complexity of the terrain shape. From the nature of the Ising model, the terrain becomes flat when the interaction parameter is large and uneven when it is small. The evaluation experiments show the effectiveness of the terrain parameterization for curriculum reinforcement learning.