Language Education & Technology
Online ISSN : 2185-7814
Print ISSN : 2185-7792
ISSN-L : 2185-7792
Articles
Transformations of Number of Words and Phrases Signaling Supporting Details: Potential Variables for Automated Rating
SPRING Ryan
Author information
JOURNAL FREE ACCESS

2023 Volume 60 Pages 1-24

Details
Abstract

Automatically calculated measures correlated with L2 productive skill can be useful in rating speaking and writing, but many can be too complicated for L2 teachers and learners to understand, some are overly topic-specific, and others exhibit too much multicollinearity to aid predictive models. Therefore, novel measures of L2 productive skills that are relatively simple and valid across topics are still required. This paper examines the use of measures based on the number of discourse markers that signal a writer is providing evidence or details for main ideas. I created an automated tool that counts these words and phrases, and then calculates several transformations, finding that counts of two discourse marker lists (SDM and IDM scores), but not transformations were predictive of L2 writing scores and TOEFL ITP® scores for L1 Japanese EFL learners across two data sets with different writing topics. Furthermore, I found that these measures could add a small amount of power to a predictive model when combined with other measures of length and complexity and that this tendency was steady across the data sets. However, the results also indicate that the two lists of discourse markers were quite different and that the creation of a new master list that combines the best elements of both would be helpful for both EFL teaching and rating in the future.

Content from these authors
© 2023 The Japan Association for Language Education and Technology
Previous article Next article
feedback
Top