Basic Research for Development of a Multimodal AR-BCI

Abstract

A Brain-Computer Interface (BCI) enables users who cannot move their own body freely to manipulate machines. Recently, a system which is a combination of BCI and Augmented Reality (AR) is proposed. This system can link real machines to augmented markers and realize an intuitive interface. However, conventional AR-BCI system uses only visual stimuli, so a user cannot operate machines behind him/her. To deal with this problem, we propose a multisensory AR-BCI using three dimensional stereophonic sound, which provides choices behind users by generating sounds including position information. First, we investigate the amplitude and latency of event-related potential (ERP) induced by three dimensional stereophonic sound. The result shows the latency of ERP is longer (650 ms) than conventional BCI. This result suggests that the feature should be extracted with a longer time window. Second, we compare the accuracy of separate classification with combined one under the mixed visual and auditory stimuli. The result shows separate classification is better than combined one (p<0.05). Therefore, our research proposes separate classification method which comes from the basic features of ERPs under multisensory AR-BCI system and shows the appropriateness of proposed method.