Evaluation Functions of Offensive Soccer Agents: Supervised Learning Based on Policy Gradients

Abstract

In the RoboCup Soccer 2D Simulation League, 22 software agents play soccer on a virtual soccer field. Each agent plays autonomously, making the 2D League a suitable test bed for studying multiagent systems. In this paper, we use an open program called agent2d. In agent2d, an agent selects an action using a search tree and an evaluation function that depends only on the ball’s position. We propose three modifications to this action selection. First, we added new terms that evaluate the state caused by an action and determine how precisely the action is executed despite interruptions by opposing players. Second, we prepared multiple sets of weight coefficients for the terms in the evaluation function. The sets of coefficients change depending on the ball’s position. Third, we developed a supervised learning system to ascertain the coefficients as the agents select actions directed by the human subjects. Our experimental results demonstrate the effectiveness of the proposed evaluation functions and learning system.