Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data
This work addresses the challenge of data scarcity and privacy concerns in remote health monitoring by improving model robustness for researchers and practitioners, though it is incremental as it builds on existing synthetic data approaches.
The paper tackled the problem of poor generalization of deep learning models for remote physiological measurement from facial videos when trained on synthetic data, by proposing methods to add real-world noise to synthetic signals and videos, resulting in a reduction of average MAE from 6.9 to 2.0 on real-world datasets.
Recent advances in supervised deep learning techniques have demonstrated the possibility to remotely measure human physiological vital signs (e.g., photoplethysmograph, heart rate) just from facial videos. However, the performance of these methods heavily relies on the availability and diversity of real labeled data. Yet, collecting large-scale real-world data with high-quality labels is typically challenging and resource intensive, which also raises privacy concerns when storing personal bio-metric data. Synthetic video-based datasets (e.g., SCAMPS \cite{mcduff2022scamps}) with photo-realistic synthesized avatars are introduced to alleviate the issues while providing high-quality synthetic data. However, there exists a significant gap between synthetic and real-world data, which hinders the generalization of neural models trained on these synthetic datasets. In this paper, we proposed several measures to add real-world noise to synthetic physiological signals and corresponding facial videos. We experimented with individual and combined augmentation methods and evaluated our framework on three public real-world datasets. Our results show that we were able to reduce the average MAE from 6.9 to 2.0.