A study of rapport estimation considering individual characteristics of speakers based on dyad interactions

Abstract

The long-term goal of this research is to realize optimal matching of dialogue partners, including the assignment of instructors for online lessons, by automatically estimating rapport, which represents "the state of connection and harmony between people" during dialogue. To achieve this goal, we propose a method that uses not only conventional multimodal features of speech, language, and vision, but also the BigFive scale of the speaker. In this study, in addition to dialogue data based on several different themes with the same speaker combination, we use the results of a questionnaire about rapport toward the dialogue partner compiled after the dialogue, and the results of a questionnaire to measure the speaker's personal characteristics. The results of the experiment showed that the BigFive Scale was more accurate than the previous studies. As a result of the experiment, we were able to confirm that the accuracy of the BigFive scale improved in cases that included either or both sincerity and openness from those that did not. This experiment provides new insights into the influence of rapport and personal characteristics on interpersonal interaction.