Knowledge Transfer for Heterogeneous Robots based on KL Divergence Regularization between Model Parameters

Abstract

This paper presents a novel knowledge transfer method for heterogeneous robot systems. Leveraging a learned model of a robot, another robot improves its learning efficacy. A main problem we tackled is to overcome discrepancy of inputs/outputs in the two systems. We introduce a method to extend neural-network model inspired by Net2Net; and derive regularization term based on Kullback-Leibler divergence between the model parameter distributions to stabilize learning process. Simulation of transferring a learned 6 DoF manipulator model to a 7 DoF manipulator model demonstrated that our method can improve sample efficiency of reinforcement learning to optimize control law of the 7 DoF manipulator.