The content analysis of the TOEIC and its relevancy to language curricula in EFL contexts in Japan

Abstract

This study particularly aims at examining the content characteristics of TOEIC and its relevancy to language curricula in English as Foreign Language (EFL) contexts in Japan. The study consists of two phases. In the first phase of the study, the content analysis of the TOEIC was conducted and the results were compared with those of the Daigaku Nyushi Center Shiken (National Center for University Entrance Examination) and two forms of a private university's entrance examination. In the second phase of the study, 136 university students' test performances on the TOEIC were examined based on Classical Test Theory (CTT) and Rasch Measurement Model (RMM). Also, factors which might affect test-takers performance were examined in terms of test method effects and item characteristics. Even though the results of the study are sample-dependent and generalizability is very limited, the findings were useful to pinpoint the strengths and weaknesses of the students' test performance. For example, low-English proficient learners of the group find the following three items difficult: (a) items that have no explicit clues in stems or response options; (b) items that include domain specific vocabulary; (c) items that require test-takers to make inferences based on the information spread across texts or other types of prompts.