The Impact of an Automated Evaluation System on Student-Writing Performance

Abstract

This study aims to examine if Japanese college students can improve their writing skills by using Criterion, an automated essay evaluation program, over a semester. The first and last essay assignments submitted online by 25 students were statistically compared in terms of fluency (essay length), accuracy (error counts), and quality (lexical richness and organization). The results revealed that the students were able to produce markedly longer essays, using more discourse elements and locating them in the appropriate places, but that there was little change in their grammatical accuracy or lexical richness over the 13 weeks. It was concluded that Criterion, to some extent, may lend itself to long-term improvement in student writing, but that it should not take precedence over classroom instruction. Based on the findings, some pedagogical suggestions are given to those who are thinking of integrating this new technology into their classes.