Judgment of Unusual Behavior in the Class Using OpenPose

Abstract

In face-to-face group classes, it is challenging for teachers to pay attention to all the students at all times. This study attempted to estimate the pose and body orientation of students by acquiring their skeletal joint points using OpenPose and judge the unusual behavior of students by capturing a video of the classroom from a bird's-eye view. Furthermore, this study introduces a method for tracking individuals using coordinates derived from each frame processed by OpenPose, along with a pose matching technique based on the Mahalanobis distance applied to seven predefined pose template vectors. Moreover, this study proposes a method for estimating the body orientation based on the angle of the line connecting the shoulders of a person. These methods were evaluated using four different videos. One of the videos was recorded in an actual elementary education classroom. Consequently, the accuracy of pose estimation exceeded 87.0%, that of body orientation estimation was 97.0%, and unusual behavior judgements were almost always made successfully.