Automatic Selection of Region of Interest for Remote Photoplethysmography from Face Video

Abstract

Pulse rate estimation based on remote photoplethysmography (rPPG) methods using face videos is in increasing demand as an alternative to contact pulse rate sensors in telemedicine and health monitoring in the COVID-19 pandemic. In rPPG methods, the pulse rate is estimated from changes in RGB signals due to fluctuations in blood flow within region of interest (ROI) on the face. However, the accuracy of rPPG cannot be guaranteed with a uniform ROI setting because of the videos taken under the uncontrolled measurement conditions, such as light sources and wearing a mask, at home or other locations. Although there are methods to heuristically determine ROI based on prior knowledge, it is difficult to estimate rPPG stably with conventional methods because of differences in measurement conditions. Therefore, the objective of this study was to automatically select effective ROI for rPPG for each frame. For the multiple regions set on the face, we trained a model that predicts whether the region is effective for rPPG using machine learning. Frequency analysis results of RGB signals in each region were used as features, and supervised learning was performed to predict whether the absolute error of pulse rate was less than a threshold value. The evaluation on videos taken in a typical laboratory environment suggests that the model would automatically select ROI that are effective for rPPG.