Host: The Japan Society of Mechanical Engineers
Name : [in Japanese]
Date : September 05, 2021 - September 08, 2021
The purpose of this study is to decode complex sounds given as auditory stimulus from the brain image acquired by fMRI. In this report, we propose a system for estimating complex sounds using 3DCNN, one of the deep learning methods. In the proposed system, to estimate complex sounds composed of multiple pitches, a two-output 3DCNN model that can identify only sound of specific pitch was created as a deep learning template for each pitches. And the scale of auditory stimulus is estimated combining the sound of specific pitch detected by the deep learning template for each pitches. As verification, 20 kinds of compound sounds with 3 sounds selected from 6 different pitches were estimated using the proposed system. The deep learning templates were created using brain image of one subject for every six pitches. As the result, the maximum estimation rate was 45.00%. And the average estimation rate was 34.72%. Although the estimation rate of each deep learning template needs to be improved, we think that the structure of superimposing a deep learning template on each pitches is useful.
