Data Sharing on Deep Reinforcement Learning for Multi-Agent Systems

Abstract

In recent years, reinforcement learning has advanced in various fields such as autonomous driving, behavior analysis in power markets, and robot control. These studies have demonstrated the utility of applying MAS (Multi-Agent System), where multiple agents cooperate and compete with each other. In the learning process within MAS, a significant challenge is that irregular fluctuations in the environment caused by the actions of other agents increase environmental uncertainty and destabilize learning. To overcome this challenge, research has been reported on methods to improve learning efficiency by effectively utilizing the experience data of other agents. Previous studies have reported cases where sharing all data improved the learning efficiency of agents in cooperative or competitive relationships, as well as cases where sharing only a portion of the experience data improved learning efficiency. This paper aims to improve learning efficiency in MAS by emphasizing the similarity between agents as a criterion for data sharing and demonstrating its effectiveness. Specifically, we propose a new method that selects shared data based on the similarity between agents and treats it as experience data that complements one's own experience. Furthermore, we demonstrate that the proposed method improves the learning efficiency of agents through simulations involving asymmetric agents.