Reconstructing Dynamic Visual Stimuli from Human Brain Activity using Deep Neural Networks

Abstract

Decoding is one of the important fields in Neuroscience, which is considered to be useful for analysis of brain function, clarification of disease and development of Brain Machine Interfaces. The purpose of this study is to decode visual stimuli from human brain activity. To reconstruct the visual stimuli, we used Neural Networks and GAN-based Neural Networks. We compared recent GAN-based models to confirm which one works the best. We also examined the difference in reconstruction quality when brain area was changed. To improve the quality of reconstruction, we combine multiple consecutive frames of human brain activity. Finally, we calculated the effect of multi-frame by quantative evaluation. The results show the effectiveness of decoding with multi-frame inputs.