Video-based Classification of General Movements Using Two-stream CNN and Recurrent Neural Network

Abstract

The general movements (GMs) assessment is used in the early diagnosis of neonatal disorders. Although this method is highly reliable, the result may vary because it relies on visual examination. In this paper, we propose a video-based classification method of GMs using a deep neural network. In the proposed method, spatial and motion features of infants' movements are extracted from video images using a two-stream convolutional neural network. The extracted features are then concatenated into a single feature vector and input to a recurrent neural network, thereby allowing the classification of the type of GMs. The experiment was conducted for 100 infants (normal GMs: 35 writhing movements, 38 fidgety movements; abnormal GMs: 27 poor repertoire of GMs). The results demonstrated that the proposed method outperformed the conventional GMs evaluation system relying on domain-dependent knowledge. Therefore, the evaluation of infants' spatial and temporal features may be effective for automatic GMs classification.