Reading-to-write (RTW) tasks have been used as integrated writing tasks because of their academic authenticity, even in Japanese classrooms. In the context of classroom assessments, accessibility to feedback and a rating scale (e.g., ease of interpretation and use) are the most important aspects. However, RTW assessments have the inevitable problem regarding the difficulty of scoring, which can influence their efficacy and feasibility in the classroom. Therefore, this study develops a classroom-oriented rating scale for RTW tasks, that is, an analytic empirically derived, binary-choice, boundary-definition (EBB) rating scale, and revises the EBB scale in a preliminary investigation using many-facet Rasch measurement (MFRM) analysis. In total, 28 writing performances elicited from an RTW task were scored by 10 raters, including four raters who participated in EBB development. The results of the first MFRM analysis showed that the first EBB rating scale functions appropriately under only two criteria. Based on these results, the first EBB was partially revised and analyzed. The revision solved the problem of the first EBB detected in the first analysis; however, several unsolved problems remained, thus suggesting room for the improvement of the present EBB rating scale.
View full abstract