Human Action Classification using Object Detection and Voice Recognition in Industrial Environments

Abstract

This paper proposes an approach that aims to detect and classify the daily actions of workers in a factory using monocular cameras. In this context, the set of actions to be detected is limited to drilling works. Towards this end, we propose an object detection method using YOLOv3 and a motion detection and classification method using the sound information included in the videos. As a result, it becomes possible to calculate the actual work time more accurately. In specific, the proposed approach is able to detect and classify various actions of work, including the detection of drilling work in industrial workspaces, such as factories, by only using one general monocular camera, with an improved efficiency.