In virtual reality, there have been many implementations using both audio and visual senses. However, mixed reality (MR), which merges the real and virtual worlds in real time, has thus far dealt with these implementations using only the visual sense. In this study, we developed an MR system that merged the real and virtual worlds in both the audio and visual senses, and in which the geometric consistencies of the audio and visual senses were coordinated. We also tried two approaches for merging real and virtual worlds in the audio sense, one using open-air headphones, and the other using closed-air headphones. The former corresponded to an optical see-through method and the latter, in the case of the visual sense, to a video see-through method.
