Introducing a Call Stack into the RGoal Hierarchical Reinforcement Learning Architecture

Abstract

Humans can set suitable subgoals in order to achieve some purposes, and furthermore, can set sub-subgoals recursively if needed. It seems that the depth of the recursion is unlimited. Inspired by this behavior, we had designed a hierarchical reinforcement learning architecture, the RGoal architecture. In this paper, we introduce a call stack into the RGoal architecture to increase reusability of subgoals. We evaluate its performance using a maze with multi-task setting. The result shows that the convergence speed improves as the maximum stack size increases.