Enhancing Interpretability in Automated Language Assessment:A Boruta-Based Approach to Key Feature Analysis

抄録

With the significant shift in the second language (L2) education landscape, developing learners’ productive skills, particularly speaking and writing, is increasingly emphasized. Consequently, there is a growing demand for efficient and objective assessment methods. To enhance the transparency and interpretability of automated language assessment, this study establishes a framework that leverages key feature analysis with the Boruta algorithm. The study aims to provide clearer insights into the linguistic features driving assessment decisions while maintaining high predictive accuracy. To achieve this, data from the Longitudinal Corpus of L2 Spoken English are used, specifically, results from the Telephone Standard Speaking Test (TSST), which is a phone-based speaking test. Boruta is applied to identify key linguistic features predictive of L2 speaking proficiency, systematically distinguishing genuine predictors from statistical noise. Focusing on TSST Levels 3-6 (CEFR A2-B1; n = 821), Boruta’s effectiveness in identifying a concise and meaningful feature set is demonstrated. The model achieves a 71.36% mean accuracy and 0.651 correlation (p < .001) with human ratings, thus revealing distinct patterns in linguistic feature use across proficiency levels. Features such as vocabulary, grammar, syntax, and discourse exhibit particularly strong relationships with proficiency. Although high-performing, traditional random forest can pose challenges for objective interpretation of variable importance, Boruta allows for statistical validation of the selected features. This study contributes to a deeper understanding of L2 development and assessment by balancing accuracy and interpretability, which enhances objectivity and consistency, thereby providing pedagogical insights that are not accessible through more complex and less transparent methods.