2008 Volume 4 Issue 1 Pages 23-32
This study investigates the potential roles of generalizability theory in investigating oral performance tests. The purposes of this study are to examine the contributions of candidates, raters, tasks, and their interactions to the variance of test scores and to find an optimal number of raters and tasks of the test using generalizability theory. Sixty-one JSL teachers evaluated six Chinese students' oral test, which consisted of the three tasks. The results of the analysis indicated that the test worked well for spreading candidates out along a continuum of oral proficiency. However, with a one-rater and one-task design, some extraneous effects on test scores that could be a source of measurement error were found, and the results indicated that in order to have higher reliability, more raters and tasks are needed for the test. As the optimal number of raters and tasks, the author suggested a two-rater and two-task design for this oral placement test. The study shows that generalizability theory is a powerful tool for investigating and developing oral performance test.
