In the research area of evaluating the effectiveness of study abroad programs, adopting counterfactual frameworks such as DID, PSM or IV has been considered a valid analytical approach. Previous findings drawn based on these conventional frameworks suggest that even short-term study abroad programs have a significant effect on the improvement of TOEIC scores. However, these studies are often designed to estimate the effects with students’ prior TOEIC scores, and only controlling departmental or school affiliations, while confounding factors, particularly students’ competency such as learning attitudes as well as learning motivations, remain uncontrolled. This study attempts to extract students’ competency from high-dimensional data with a large volume of missing values in student’s test score sheets, using the Alternating Least Squares (ALS) method. Injecting the extracted competency in the subsequent regression analysis en ables us to accurately estimate the causality between the study abroad experience and the observed outcomes. Our analysis result reveals that, unlike findings from earlier studies, once students’ competency is properly controlled, the estimated effect of the study abroad programs becomes negligible with no significance. Therefore, the finding suggests that the causal effect claimed by the previous studies might be due to a bias engendered by students’ self-selection. The result also indicates that datasets readily accessible at any university, such as student test score sheets, could effectively be used for project evaluations within an institution, notably because the confounding factors are properly controlled as suggested by the current study.
