Regularizing Oversmoothing of Temporal Convolutional Networks for Action Segmentation into Human Assembly Operations

Abstract

We investigated a (supervised) deep learning model for the automatic segmentation of manufacturing video data into a sequence of assembly operations. In the action segmentation for general human behavior, various temporal convolutional network (TCN)-based methods have been proposed and demonstrated to have stable performance using temporal features from captured extensive fields of image frames. However, they often make it difficult to detect unusual actions occurring in short durations, such as skipping assembly operations. In this paper, we address this drawback in the existing TCN model by introducing two techniques of differentiable group normalization and jumping knowledge to relax the oversmoothing effect. Then, we empirically show the performance of the proposed TCN variations using a benchmark for operational assembly work segmentation. We also present a preliminary result on irregular work detection.