Assessment of Good Essays in Japanese as a Second Language by Japanese Teachers : Exploring the Process of and the Decisive Factors in Essay Ranking

Abstract

This study aims to clarify what teachers of Japanese as a second language consider a "good essay" for the purpose of improving writing education for students of Japanese. Our study involves analysis of both the teachers' rating processes and the rating criteria. We conducted research on ranking. Two types of six essays were ranked separately by 10 experienced Japanese teachers using holistic schemes. Raters' think-aloud protocols were recorded and analyzed along with a questionnaire. The results indicated that "fulfillment of task," "clarity of claim," "originality of content," "objective support from broader perspectives," "text organization," "discourse development" and "expressiveness of Japanese" were the decisive factors. However, there appeared to be no common understanding on the priority of each factor among raters. For essays with a good balance of each factor, the assessment results were agreed upon. Some essays, however, despite having failed to fulfill the task, or lack of such balance, were ranked high in holistic schemes because "academism" (academic atmosphere) was perceived in the supporting ideas or throughout the essay, as well as a high level of expressiveness. The study confirmed the necessity of a more common understanding, including scoring rubrics and scoring procedures, for writing assessment among Japanese language educators.