Influence of Simulated Contact Sound on Haptic Sensory Illusion in MR Environment

Abstract

Cross-modality is applied as a haptic feedback method in a virtual / mixed reality (VR/MR) environment using a head mount display (HMD). Such a haptic feedback method based on cross-modal illusion is seen as a method reducing burden on the user. Particularly, there are reports of haptic sensory illusion that occurs when a virtual object is touched in MR environment which can be applied to haptic feedback methods. But there is a problem that the haptic sensory illusion obtained from this method is weak. In anticipation of a more reliable haptic feedback method, we focused our research on hearing as it is a modality co-occurring with the sense of touch. In this paper, we aim to understand the influence of auditory stimuli on haptic sensory illusions when touching virtual objects in an MR environment. In particular, we considered the influence of adding simulated contact sound when a user touches a virtual object with his or her palm on the magnitude and impression of the haptic sensory illusion. Regarding magnitude, the haptic sensory illusion was increased from adding auditory stimuli. Regarding impression, a soft impression changed to a hard impression by adding auditory stimuli, in one of the trials.