This paper introduces the writing section of the National English Ability Test (NEAT) developed for college entrance exam in Korea. It will discuss the nature of the writing tasks of the NEAT, rating domains and rater training procedure.

View full abstract

This study investigates how distractors function in multiple-choice listening tests. Conventionally, each distractor is evaluated by its attractiveness. In other words, distractors that can plausibly be chosen by test takers are considered to perform well on the test. On the other hand, distractors that are chosen by few test takers are recognized as performing poorly. Considering that test takers have to choose only one among several options, it can be assumed that some unselected distractors may have in fact performed adequately. Therefore, it is prudent to independently analyze the attractiveness of distractors. A total of 75 Japanese university students evaluated their confidence in selecting both correct and incorrect answers. The results indicated that (a) the least chosen distractors were not always the least attractive, (b) less proficient listeners were more likely to be allured by distractors, and (c) more proficient listeners were more likely to answer with higher confidence. The researcher explains the process of eliminating distractors and reevaluates the unselected distractors.

View full abstract

In construction of Sentence Repetition Test (hereafter, SR), both sentence length and pause between the presentation of the target sentence and its repetition, have been found to play an important role in SR performance. The present study investigated how these two factors influence the task difficulty and the concurrent validity of SR with TOEIC scores. In order to achieve these aims, 79 Japanese high school students participated in this study and undertook the English SR under two conditions-with and without a pause. The SR sentences were varying lengths. The results showed that, while long sentences were more difficult to recall correctly than short ones, the pause made the short sentences more difficult and long sentences easier to repeat. Multiple regression analyses indicated that the performance under the most difficult condition, where the test-takers repeated long sentences without any pause, showed the highest concurrent validity with TOEIC. The implications of the present findings, along with some suggestions for further research, will be discussed to calibrate SR for second language test-takers.

View full abstract

The purpose of the present study is to examine the appropriateness of item response theory (IRT) to language testing. Although it has been discussed by many researchers (e.g., Henning, 1992; Blais & Laurier, 1995) for a long time, its appropriateness has not been demonstrated using language testing data, especially listening test data. Therefore, the dimensionality of the listening test data was examined by several approaches in the present study. The 50 questions in the listening section of the TOEFL Sample Test (6th edition) were administered to 392 students as a part of their usual General English classes. The data were analyzed using two approaches: the factor-analytic approach and the principle component analytic approach. The analyses of the 30 questions in Part A showed less possibility of the existence of a second dimension. As for the analyses of the 20 questions in Part B and C, however, two out of the five analyses affirmed the possibility of a second dimension. These mixed results suggested that multidimensionality may be detected depending on the methods used. Moreover, it was found that the different tasks tended to measure different dimensions, even though they seem to measure the same language skill. In addition, there was a fairly large amount of unexplained variance in the data. It is possible to say that there was a great deal of noise in the data that could not be aligned along dimensions. An implication is that tests using IRT should be more construct-valid. Lastly, it is observed that unidimensionality is a continuum rather than one position of a binary variable. Its appearance depends on the methods used to seek it.

View full abstract