Multilevel Classification of Drowsiness States Using Multimodal Physiological Signals

Abstract

Detecting drivers' drowsiness with high accuracy and fine granularity is essential to ensure road safety. With the growing popularity of wearable devices, various physiological signals have become accessible, enabling drowsiness detection anywhere and at any time. Recent studies have achieved multilevel drowsiness detection, identifying up to eight drowsiness states using a single ECG signal. However, the effectiveness of using multiple physiological signals remains unclear. To address this, this study conducted four types of drowsiness detection, each with varying granularity, by utilizing ECG and EMG signals from the DROZY dataset. We first built models for each single modality using CNN and LSTM, optimizing model parameters to identify the best-performing models for each modality. We then built a multimodal model by concatenating the best-performing models for the two modalities. As a result, for fine-granularity drowsiness detection, using multimodal signals outperformed detection only using a single modality of signals. In addition, the optimized model for multilevel drowsiness classification is also identified.