Evaluating a Virtual Agent Who Responses Attentively to Multiple Players in a Quiz Game

Abstract

In developing an entertainment application like a game, evaluating how the players actually interact with the system is an essential issue for its further improvements. This paper proposes and evaluates a quiz game agent who is attentive to the dynamics of multiple concurrent participants (players). The attentiveness of this agent is meant to be achieved by an utterance policy that determines the nature of the utterance and whether, when, and to whom to utter. Two heuristics are introduced to drive the policy: the interaction atmosphere (AT) of the participants and the participant who tends to lead the conversation (CLP) at a specific time point. They are estimated from the activeness of the participants' face movements and acoustic information during their discussion of the answer. In order to prevent the inherent drawback of a 2D agent that makes it difficult for multiple concurrent users to distinguish the focus of its attention, a physical pointer is also introduced. This system is then evaluated from three aspects, participants' own subjective measurement by questionnaires, participants' implicit attitude by an external measuring test, and from third-person view by video data analysis. The joint results of the experiments indicated that the methods for estimating AT and CLP worked. The participants pay more attention to the agent and participate in the game more actively if the indication of the pointer is more comprehensive.