Adaptive selection of auxiliary tasks in UNREAL

Abstract

Deep reinforcement learning has a difficulty to solve a complex problem because such problem consists of a larger state space. To solve this problem, Unsupervised Reinforcement learning and Auxiliary Learning (UNREAL) has been proposed, which uses several auxiliary tasks during training. However, all auxiliary tasks might not perform well on each problem. Although we need to carefully design these tasks for solving this problem, it requires significant cost. In this paper, we propose an additional auxiliary task, called auxiliary selection. The proposed method can adaptively select auxiliary tasks that contributes the performance improvement. Experimental results with DeepMind Lab demonstrate that the proposed method can select appropriate auxiliary tasks with respect to each game tasks and efficiently train a network.