Hierarchical Reinforcement Learning using Fractal Reservoir Computing for Quadrupedal Robot

Abstract

Catastrophic forgetting is one of the most challenging problems of (deep) neural networks, but autonomous robots, which would acquire many tasks in real life sequentially, require to resolve or mitigate it. Modular networks are expected to mitigate this problem since it can exploit different modules for respective tasks. However, this approach would waste the learnable parameters due to duplication of common tasks in given tasks. Furthermore, if given tasks, e.g., locomotion control of legged robots, are with high state-action spaces and are difficult to be learned, exploration is required for a long time, which causes the catastrophic forgetting. Hence, this paper proposes the way to divide the locomotion control tasks modularly and hierarchically. To this end, fractality of fractal reservoir computing is utilized so as to transfer the learned knowledge of one leg control to the other legs control.