Does controller sound contain valuable information for video game scene analysis? Case study by character identification of Super Smash Bros. Ultimate

Abstract

Understanding of gameplay can enhance the experience and entertainment of video game. In this study, we propose to utilize the sound generated by a controller for analyzing the information of gameplay. Controller sound is a user-friendly feature related to gameplay because it can be very easily recorded. As a first step of the research, we performed identification of characters of Super Smash Bros. Ultimate only from controller sound as an example task for examining whether controller sound contains valuable information. The results showed that our model achieved 79% accuracy for identification of five characters only using the controller sound.