How Multiple-Choice 1-Blank Partial Dictation Items Behave

Abstract

Segmenting a stream of continuous speech into recognizable words presents one of the biggest challenges to novice EFL listeners. One straightforward method of checking how good a learner is at this task is dictation. Two members of the dictation family are partial dictation and multiple-choice (MC) dictation. Combining these two techniques, the author and his colleagues devised multiple-choice 1-blank partial dictation (MC1PD). An earlier study that examined the nature of MC1PD produced partly expected and partly unexpected results. Comparison of MC1PD and its open-ended (OE) counterpart indicated that success on an OE item usually entails success on its MC counterpart but not vice versa, which had been expected. Comparison of MC1PD and TOEIC® scores indicated that MC1PD correlated more strongly with TOEIC® reading than with TOEIC® listening, which had been unexpected. This unexpected pattern had to be attributed to random noise caused by the sample for which TOEIC® scores were available being small. In addition, a number of distractors of MC1PD items used in the earlier study were subsequently found to be non-functional. The present study addressed the following research questions: (a) Will replacing non-functional distractors of MC1PD with new options better the items’ psychometric performances? (b) Can the expected relationship between MC1PD items and their OE counterparts observed again using the revised MC1PD test? (c) Can reasonable and interpretable relationships be found between MC1PD scores and TOEIC® scores, using the revised MC1PD test? A revised MC1PD test was prepared by updating 14 distractors in the MC1PD test used in the earlier study. A total of 115 Japanese EFL university students took an OE partial dictation test and the revised MC1PD test, with a one-week interval between them. A subsample (n = 85) also sat for institutional TOEIC® tests. Classical and Rasch analyses of the data indicated that the answers to the three research questions were all affirmative.