Curiosity Driven by Self Capability Prediction

Abstract

Reinforcement learning is a powerful method to solve tasks using a reward signal; however, it struggles in sparse reward scenarios. One solution to this problem is the use of reward shaping but, it requires complicated human engineering in complex environments. Instead, our solution relies on exploration driven by curiosity. In this paper, we formulate the curiosity as the ability of the agent to predict its knowledge about the task. The prediction is based on the combination of intermediate goals and deep learning. Our end-to-end method scales to high-dimensional state spaces such as images. As proof-of-concept, we present a preliminary implementation of our algorithm using only raw pixels as input.