Evaluating the Effectiveness of an English Learning Recommendation System Using Causal Inference

Abstract

In recent years, educational reforms have highlighted the growing importance of English language learning, emphasizing the need to acquire English proficiency required to pass the Eiken Test in Practical English Proficiency (Eiken). In this study, we developed a recommendation system that adjusts question content based on users' learning volume and answer tendencies and applied it to English learning aimed at passing the Eiken test. Using the learning log data and Eiken pass/fail results obtained through this system, we evaluated its effectiveness using T-learner, a causal inference method. The analysis suggested that the use of this system could improve the probability of passing the Eiken test. Future studies need to verify whether similar effects can be observed in different periods and under various learning conditions. The results of the analysis suggested that the use of this system may improve the probability of passing the EIKEN. Further studies are needed to verify the effectiveness of this system in English language learning by using other log data.