ARELE: Annual Review of English Language Education in Japan
Online ISSN : 2432-0412
Print ISSN : 1344-8560
ISSN-L : 1344-8560
Research Articles
Reliability and Optimal Designs for Measuring Accuracy in L2 Writing With a Weighted Clause Ratio
Hideaki OKA
Author information
JOURNAL FREE ACCESS

2021 Volume 32 Pages 49-64

Details
Abstract

  Prior research has proposed a new index for assessing written accuracy known as weighted clause ratio (WCR). However, the descriptors used to determine WCR scores are vague and open to multiple interpretations. Additionally, prior research has not investigated the impact that various factors (e.g., rater) might have on the WCR score variances and reliability. Optimal measurement designs have not also been investigated. Thus, this study revised the rating scale descriptors used to determine WCR scores and examined the index’s reliability using generalizability theory. The impact of factors that affect the score variances and optimal measurement designs were also investigated. English essays written by 100 Japanese English language learners from the International Corpus Network of Asian Learners of English were evaluated for this purpose. The results indicated that the reliability of the evaluation was high (G coefficient = 0.91), although only some factors affected the variances of WCR scores. Consequently, optimal measurement conditions were found under which written accuracy can be reliably measured using WCR. Results indicate that the revised rating scale of WCR can be applied to essays written by Japanese EFL learners. The study results can also serve as a guideline for assessing accuracy using WCR.

Content from these authors
© 2021 The Japan Society of English Language Education
Previous article Next article
feedback
Top