The Impact of a Music Co-Listening Agent Expressing Chord Perception on User Empathy

Abstract

In this study, we propose a music co-listening agent that expresses pleasure/displeasure in response to the brightness of chords, tension in response to unstable chords, and a sense of relief in response to the progression from unstable to stable chords. First, we examined the appropriateness of the facial expressions of the chord co-listening agent and, based on this, their effects on users’ sense of empathy toward the agent and on the chord-listening experience (Experiment 1). The results showed that (1-a) with respect to the appropriateness of facial expressions, the agent’s emotional transitions enhanced the naturalness of both the chord progressions and the facial expressions, indicating that the proposed method produced facial expression changes that were appropriate and consistent with the chords. In addition, (1-b) the agent was perceived as another co-listener, and the results suggested that the agent facilitated users’ sense of empathy, confirming its effect on enhancing user empathy. Furthermore, (1-c) co-listening with the agent improved the enjoyment and pleasantness of listening to chord progressions. Next, we investigated the effects of individual musical experience, the appropriate intensity of the agent’s facial expressions, and the effectiveness of relief expressions through sighing behavior (Experiment 2). The results showed that (2-a) the sigh expression conveyed an impression of relief, as intended by the proposed system. Moreover, (2-b) combining strong facial expression intensity with sigh expressions enhanced the impression that the agent’s emotional transitions were aligned with the chord progressions, led users to perceive the agent as a co-listener, and facilitated a sense of empathy toward the agent. Finally, (2-c) these effects were found to be more pronounced in participants with musical experience than in those without musical experience.