Action evaluation of multiple soccer players in continuous state space based on deep reinforcement learning

Abstract

Advances in measurement technology have made it possible to acquire various data during a match, and advanced data analysis is being used to plan team tactics, and evaluate and coach players. Analysis of invasive sports such as soccer is difficult because the game situation is continuous in time and space, and multiple agents individually recognize the game situation and make decisions. In the previous study using deep reinforcement learning, which is one of the representative agent modelings, they have often considered the team as one agent and evaluated the players and teams who hold the ball in each discrete event. Therefore it was difficult to evaluate the behavior of multiple players, including players far from the ball, in a spatio-temporally continuous state space. In this study, based on a deep reinforcement learning model with a discrete action space in a continuous state space that mimics Google Research Football (a reinforcement learning platform for soccer), the actions in actual games are evaluated by estimating the action-value function of multiple players. In the experiment, the calculated player evaluation index was verified using the data of one season of a team in the J-League.