Situational Task Ratings by Non-specialists

Validation of Japanese Speaking Test for Placement

Abstract

In this study, 30 raters other than language teachers participated in an experiment where they were asked to rate the speaking level of learners of Japanese in situational tasks, in order to validate a speaking test designated as STAR (Speaking Test of Active Reaction), developed as a placement test by Beuckmann et al. (2020). From the analysis of the results, despite a possible insufficient discrimination of the intermediate levels, STAR proved to have sufficient ʻusefulnessʼ (Bachman & Palmer, 1996), since STAR shows it has good reliability, validity, and practicality. The analysis of the ratersʼ comments also found that: 1) naturalness and fluency are characteristics of the advanced level; 2) the perspectives used for content, text type, and interpersonal consideration differ between the intermediate-low and the intermediate-middle on the ratings; and 3) the perspective for grammar differs between the intermediate-middle and the intermediate-high. Furthermore, the analysis also revealed that there were many positive comments for the intermediate-low level as they seem to be compared with the elementary level, while there were many negative comments for the intermediate-middle level relative to the advanced level.