Application of deep reinforcement learning to networked control systems with uncertain network delays

Abstract

Networked control systems (NCSs) have been attracted much attention thanks to the development of network technology. There are network delays caused by data transmissions in NCSs. These network delays may degrade control performances. In general, the network delays may fluctuate randomly and it is difficult to identify their probability distributions. Moreover, it is difficult to precisely identify models of plants. Thus, we propose a design method of networked controllers using deep reinforcement learning (DRL) taking network delays into consideration. Additionally, we consider the case where sensors cannot observe all state variables of plants. We introduce an extended state and propose a DRL-based controller design method.