Development and evaluation of system guiding memory based on emotion estimation from dialog

Abstract

Memory plays an important role in human mental health. Therefore, the effectiveness of emotional interventions for individuals depends on what memories are recalled. In order to effectively intervene on emotions through memory, it is necessary to estimate emotions at each time point in real time. In this study, we develop a memory guide system that combines a model of memory built by the human cognitive architecture with emotion estimation. The system presents photos to the user and prompts to storytell the memory from the photos. The parameters of the memory model are adjusted by estimating the user's emotion from the audio data obtained from the storytelling. The parameters of the model are activation, which is related to the intensity of the memory, and utility, which is assigned to the memory retrieval rules. These parameters are mapped to the user's arousal level and emotional valence estimated from the voice data. In this way, the system prompts memory by sensing the current user's emotion. We conducted an experiment using this system with participants recruited through crowdsourcing. The results of manipulating the type of interface and the parameter adjustment partially showed the linkage between the model and the participants.