Building a Random Forest model for predicting pass/fail in the National Pharmacist Examination using low-grades performance data, exploration of predictors, and evaluation of prediction accuracy

Abstract

Some private Japanese universities have recently faced low student pass rates on the National Pharmacist Examination within the standard six-year pharmacy program. As a result, this study employed data on first- to third-year students’ academic performances to identify the students requiring learning support and apply a model Random Forest algorithm to predict whether the students would pass the National Pharmacist Examination. The average decrease in the Gini coefficient of the model using Grade Point Average (GPA) revealed that grades in the second semester of the third year were most significant for prediction. The receiver operating characteristic area under the curve (ROC_AUC) for the model using the GPA of first to third year of students enrolled in 2015 to predict the pass or fail rates of the National Pharmacist Examination for students enrolled in 2016 and 2017 was 0.86 and 0.75, respectively. Similarly, for the model employing specialized subject regular examination scores in the second semester of the third year, ROC_AUC was 0.80 and 0.74, respectively. The ROC_AUC of the model using specialized subject regular examination scores for the first semester of the second year was 0.69 and 0.67. These findings suggest that constructing a model to predict a pass or fail on the National Pharmacist Examination using GPA data from the first to third-year academic performance may help identify the students who require learning support early on in the program.