Validating the Effectiveness of a Meeting Summarizing Browser that Implements an Important Utterance Detection Model based on Multimodal Information

Abstract

This paper proposes a multimodal browser designed to t the characteristics of an important utterance detection model based on co-occurrence of verbal and non-verbal information expressed by discussion participants ^[1]. The multimodal browser aims to support its users to understand discussion accurately by implementing functions such as visualizing important utterances and displaying verbal and nonverbal information. We conducted an experiment to compare the multimodal browser condition with control conditions in terms of the degree of understanding and task completion time, and understanding of the roles of participants in discussions. In control conditions, we built two browsers: a text-based browser that doesn’t t to the characteristics of the model, and a simple browser that is similar to a simple movie player without any support for understanding a discussion. We found that the multimodal browser enabled the users to accomplish the correct understanding of discussions even in a shorter time, compared to the simple browser. In addition, it enabled the users to accomplish more accurate understanding than the text-based browser. Furthermore, in terms of understanding the role of discussion participants, in some tasks, the multimodal browser enabled the users to understand the role of discussion participants more accurately than the text-based browser.