Action as vector to the goal state in latent space

Abstract

In reinforcement learning, action is treated as a point in the action space, with little emphasis on the design of the action space. In contrast to the existing reinforcement learning frameworks, we consider action as the amount of change in the latent space to reach the target state, referring to the human action process, and define this as latent action. We propose a representation learning method using Predictive Variational Autoencoder which enables that taking latent action to minimize the distance to the goal state in the latent space corresponds to the optimal action in the actual input space. We verify by experiments that action selection by latent actions using Predictive Variational Autoencoder can achieve more stable control compared to the method which uses Variational Autoencoder for current observation and selects actions based on errors from the control goal in the input space. And we discuss possible issues in extending the action selection method using latent actions.