Mirroring effects of pupillary responses in the speech-driven embodied listening system

Abstract

It is known that human pupil is related to own affects or interests and becomes more attractive when it is expanded. Focusing on effects of the dilated pupil in human communication and interaction, we analyzed the relationship between the speaker’s speech and his/her own pupil response and confirmed that the pupil dilates in synchronization with the speech. Based on the analysis, we developed the pupil response interface that expresses various pupil responses such as dilated or contracted pupil by speech input. In addition, we developed the speech-driven embodied listening system by mirroring of pupil response in synchronization with the speaker’s speech. This system expresses a positive listening attitude by the pupil response and the nodding response. In this study, we evaluated impressions that the system mirroring the pupillary response acted as a listener. The effectiveness of the system by mirroring the pupil response was demonstrated by sensory evaluations.