Punishment Shaping Algorithm for Multi-Agent Reinforcement Learning with Primitive Risks

Abstract

Multi-Agent Reinforcement Learning (MARL) is a framework that utilizes reinforcement learning to simultaneously learn policies for multiple agents, such as robots, within the same environment. One concern with reinforcement learning is that stochastic behavior during learning can lead to risk for the agent. In the context of MARL, appropriately avoiding risks such as collisions between agents, is necessary. This study aims to achieve mutual risk avoidance by evaluating the overall risk of multi-agent system (MAS) when each agent has its primitive risk. The focus is on the differences in the nature of rewards and penalties in MAS. The proposed method is designed based on risk evaluation using maximum punishment. Simulation results demonstrate that the proposed method achieves more advanced risk avoidance compared to risk evaluation based on mean punishment for all agents.