A Method of Knowledge Transfer by State Value for Reducing Learning Time

Abstract

This paper proposes a knowledge transfer method based on state value for reinforcement learning (RL) agents. It has a fundamental problem that RL which is the one of machine learning techniques needs a lot of time or the number of trials because the agents acquire appropriate skills through trial and error in order to solve a task. Transfer learning (TL) allows the agent to transfer knowledge which is acquired by itself in other tasks, or previous knowledge to solve a target task. So, TL for RL is able to speed up the learning than simple RL. Our proposed method transfers both state value and a new policy which is given by state values of two selected knowledge to as initial knowledge for an unknown state. The effectiveness of the proposed method was verified with the simulation of the reaching problem for a multi-link robot arm. The proposed method has reduced the learning time 40% than the conventional method.