Building a Machine Learning Model to Analyze Evaluators’ Rating in Skill Assessment of Project Management Training Board Game Participants

Abstract

To cultivate top-tier project managers, we have created a board game specifically designed for project management training. Hundreds of participants have already benefited from this game, receiving valuable feedback on their practical skills. However, the current evaluation method relies heavily on seasoned facilitators. For more consistent and long-term participant development, we aimed to standardize the skill assessment process.

Initially, we conducted a workshop with three veteran project managers and a psychology expert to identify the evaluation criteria and essential skill sets from a problem-solving perspective. We then used statistical methods to analyze rating inconsistencies attributed to the evaluator’s field of experience.

As a next step, we developed a machine learning model to predict ratings from evaluators. This model uses data from a digital version of our training board game. By examining the decision tree within the model, we were able to identify the specific variables causing variations in the evaluators’ ratings.