A Tailored Cloze Test: The Effect of an Increased Potential for Alternative Answers on EXACT and SEMAC Scoring

抄録

Previous studies on the results of doze tests reported a veiy high correlation between the exact word scoring method (EXACT) and the semantically acceptable scoring method (SEMAC), ranging in the .90s (e.g., Brown, 1980; Ilyin, Spurling, & Seymout 1987; Owen, Rees, & Wisener, 1997). The present study tailored a doze test that would raise the potential for subjects (158 Japanese high-novice and low-intermediate EFL learners) to produce more acceptable altemative answers (AAAs) than usual to examine if such an intentionally biased doze still yields correlations in the same range as above. In this study the results of the two scoring methods were slightly less correlated than those of previous studies, but they were nevertheless sufficiently high (e.g., the calculated Spearman' rank-order correlation coefficient was .84). SEMAC was superior to EXACT in achieving score dispersion and differentiating the subjects and proficiency levels, whereas EXACT made the test too difficult in terms of Item Facility (IF). The reliability coefficients for SEMAC had higher values than those for EXACT. Assessments were made to decide which scoring method should be employed in the light of pedagogical uses for the test, such as measuring language proficiency accurately and ESL/EFL placement testing.