Nonlinear Theory and Its Applications, IEICE
Online ISSN : 2185-4106
ISSN-L : 2185-4106
Regular Section
Continuous deep Q-learning with a simulator for stabilization of uncertain discrete-time systems
Junya IkemotoToshimitsu Ushio
Author information
JOURNAL FREE ACCESS

2021 Volume 12 Issue 4 Pages 738-757

Details
Abstract

A simulator that predicts the behavior of a real system is useful for reinforcement learning (RL) because we can collect experiences more efficiently than through interactions with the real system. However, in the case where there is an identification error, the experiences obtained by the simulator may degrade the performance of the learned policy for the real system. Thus, we propose a two-stage practical RL algorithm using a simulator. In the first stage, we prepared multiple premised systems in the simulator and obtained approximated optimal Q-functions for these systems. In the second stage, we represent a Q-function for the real system as an approximated linear function whose basis functions are the approximated optimal Q-functions pre-trained in the simulator. The approximated linear Q-function is learned through interactions with a real system.

Content from these authors
© 2021 The Institute of Electronics, Information and Communication Engineers
Previous article
feedback
Top