Brain-integrated BERT: Making the behavior of BERT more brain-like via brain-activity prediction

Abstract

Recent natural language processing (NLP) technology has developed greatly. However, while its performance has improved, its human-like behavior can still be improved. This study aims to endow more human-like behavior with NLP technology by using our method, which integrates brain information into artificial intelligence via brain-activity prediction. We applied this method to a NLP model, BERT, and referred to it as brain-integrated BERT (Bi-BERT). This method first constructs a model for predicting fMRI signals during watching movies using BERT's internal representations obtained from scene descriptions of the movies. The trained model can predict fMRI signals from arbitrary descriptions with no additional brain measurements. We validated Bi-BERT by comparing it with standard BERT in terms of estimation accuracy, representational similarity with brain, and individual-difference reflection during estimating movie-evoked human cognition from scene descriptions. We found that Bi-BERT showed higher performance than BERT in all the comparisons, suggesting that our method for brain-information integration is useful for furnishing more brain- and human-like behavior to NLP technology.