Integration processes of audiovisual spatial information during long-term adaptation to left-right reversed audition

Abstract

Experiments of participants living in an unusual environment tell us an adaptive mechanism of multisensory integration that could be a key to improve cybersickness. As less attempts have been made to examine adaptation to unusual audition as compared with unusual vision, we developed left-right reversed stereophonic audition using wearable devices and asked participants to wear them continuously for about a month. Psychophysical experiments using the ventriloquism effect showed perceptual reversals of audiovisual spatial integration. Moreover, MEG responses under the audiovisual matching task revealed two steps of adaptation: early adaptation occurring in superior temporal sulcus presumably related to perception and late adaptation occurring in auditory cortex presumably related to behavior. We conclude that these non-linear processes of adaptation achieve optimization of audiovisual spatial integration.