汎用言語モデルBrainBERTを用いた言語刺激下の脳内状態推定

抄録

Currently, many researchers have used language models to achieve excellent results in various fields, such as understanding the semantics of text and extracting multimedia information like videos. Furthermore, many investigations have also been conducted to capture the generative correspondence between text and the brain. In this paper, we constructed a model based on the correspondence between brain activity data and semantic representation by BERT, called BrainBERT. The BrainBERT was used to build an encoding model between text and brain activity states, and to estimate the brain activity. In brief, we have achieved the two primary achievements of this research. 1) We verified the superiority of the BrainBERT model for brain signal extraction compared to the other 20 popular language models. 2) Using visualization tools such as PyCortex, we visualized the correlation of brain activity data according to the regions of interest in the brain.