Human Behavior Recognition Algorithm Based on Multi-Modal Sensor Data Fusion

抄録

With the advances in artificial intelligence and computers, sensor-based human behavior recognition technology has been gradually applied to many emerging cross-cutting fields such as smart healthcare and motion monitoring. First, we design a deep learning model for human behavior recognition single mode based on lifting wavelet transform (lifting scheme convolutional neural networks-gated recurrent unit, LSCG) to address the problem of inaccurate and insufficient feature extraction from sensor data in the human behavior recognition network model. The structure of the LSCG network model consists of a wavelet decomposition module and a feature fusion module. Then, we further address the limited ability of a single modality for human behavior recognition by designing a multimodal human behavior recognition model based on the LSCG model (multimodal lifting scheme convolutional neural networks gated recurrent unit, MultiLSCG). The structure of the MultiLSCG network model consists of a feature extraction module and a multimodal feature fusion module. The feature extraction module consists of the LSCG model, which enables the model to extract features from different modal human behavior data. The multimodal feature fusion module enables the model to obtain more features from the multimodal behavior signals by extracting the global feature information of the human behavior signals and the local feature information of the human behavior signals. Finally, the experimental results show that in the public dataset OPPORTUNITY, the accuracy of the motion pattern dataset reaches 91.58%, and the accuracy of the gesture recognition dataset reaches 88.53%, which is higher than the existing mainstream neural networks, on the UCI-HAR and WISDM data sets, the accuracy of our proposed model reached 96.38% and 97.48%, which further verified the validity and applicability of our proposed model.