マルチエージェントシステムのための信頼度を用いた強化学習

—相補ゲームにおける競合回避行動の獲得—

抄録

This paper proposes a new reinforcement learning approach for acquiring conflict avoidance behaviors in multi-agent systems. Multi-agent systems are able to establish orderly systems autonomously through interaction with autonomous agents. We expect to be able to construct flexible and robust systems for the environmental changes by using multi-agent system approaches. However, it is difficult for designers to preliminarily embed appropriate behaviors to avoid conflict because complex dynamics emerges by interaction between many agents. We apply the proposed method to the narrow road problem that many agents go by each other in a narrow road, and verify the effectivity of the proposed method. In the narrow road problem, it is the optimal strategy that an agent selects going forward and another agent selects giving way. However, it is difficult for agents to decide which strategy to select because they cannot predict other agents' behaviors beforehand. The proposed method can differentiate into agents preferring to go forward and agents preferring to give way, by using Q-learning that can adjust discount rates. We solve conflict problems in multi-agent systems through autonomous functional differentiation of many learning agents. Through experimental results, we showed that agents differentiated into two type of agents, and acquired stable conflict avoidance behaviors with high probability than a conventional Q-learning.