SRSP: Sentiment-Response Simultaneous Prediction model

Abstract

Nonverbal information is sometimes more important than words in human communication since it provides emotional information of speakers on current dialogues. This paper proposes the SRSP (Sentiment-Response Simultaneous Prediction model) that has the following two functionalities. (1) It learns human sentiment from the analysis on multi-modal aspects on user utterances. (2) It applies the sentiment analysis results predicted by the above function in learning response selection. Our evaluation used the CMU-MOSEI dataset and showed that SRSP achieved higher response selection accuracy compared with a method that learns response selection without sentiment analysis.