Remote Pulse Estimation in the Presence of Face Masks
This research addresses the critical problem of accurate contactless health monitoring for individuals wearing face masks, which is highly relevant in current public health contexts.
The widespread use of face masks due to the COVID-19 pandemic significantly increased the mean absolute error of heart rate estimation by over 80% using existing remote photoplethysmography (rPPG) methods. By augmenting unmasked face videos with synthetic face masks, the researchers improved performance and reduced the accuracy gap between masked and unmasked pulse estimation.
Remote photoplethysmography (rPPG), a family of techniques for monitoring blood volume changes, may be especially useful for widespread contactless health monitoring using face video from consumer-grade visible-light cameras. The COVID-19 pandemic has caused the widespread use of protective face masks. We found that occlusions from cloth face masks increased the mean absolute error of heart rate estimation by more than 80\% when deploying methods designed on unmasked faces. We show that augmenting unmasked face videos by adding patterned synthetic face masks forces the model to attend to the periocular and forehead regions, improving performance and closing the gap between masked and unmasked pulse estimation. To our knowledge, this paper is the first to analyse the impact of face masks on the accuracy of pulse estimation and offers several novel contributions: (a) 3D CNN-based method designed for remote photoplethysmography in a presence of face masks, (b) two publicly available pulse estimation datasets acquired from 86 unmasked and 61 masked subjects, (c) evaluations of handcrafted algorithms and a 3D CNN trained on videos of unmasked faces and with masks synthetically added, and (d) data augmentation method to add a synthetic mask to a face video.