Context effects on Japanese voice misperceptions

Abstract

The McGurk effect occurs when inconsistencies between a person’s voice and facial movements cause misperceptions among listeners. In this study, we investigated whether these misperceptions are affected by the context in which the stimuli are experienced. We used sentences with a target word that was either a meaningful (word sentence) or meaningless (non-word sentence) three-syllable term. The speakers’ voices and facial movements during the second (target) syllable of the target word involved the sounds /ba/, /ga/, or /da/, which were combined independently. Participants reported what they heard during the combined voice-movement stimuli. Incongruity between target syllables in the word sentences and voices indicated a high error ratio with regard to the voices. In addition, congruency between target syllables in non-word sentences and voices also indicated a high error ratio for voices. When the sentences and facial movements were meaningful, many responses matched the facial movements shown, beyond the classic McGurk effect. Completing the Japanese sentences with misperceptions was caused by higher-level cognitive processing in order to understand the sentence.