On relationship between modalities and language cognition

Abstract

Language contains not only speech, but also movements such as signs and fingerspelling. To investigate the influence of the movement and the speech including duration and the letters number on cognition, this research conducts an oddball task to record Event-related Brain Potentials (ERPs). The oddball task sets up 3 conditions: letters, speech and Japanese Sign Language (JSL) fingerspelling. In the letters and speech conditions, 2-mora and 5-mora words are adopted to search for the characteristics of cognitive timing. Then in the JSL fingerspelling condition, voiceless and voiced kana-syllable are adopted. In the results, P300 in letters condition was confirmed about 300 ms after the timing of presentation regardless of the number of letters. Meanwhile, P300 in speech and fingerspelling conditions was confirmed about 300 ms after the point of change in sound of speech and action of fingerspelling. In ERPs research on sign language, it is necessary to consider the timing of presentation and the point of change. Therefore, further analysis of the result is required.