PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurement
This work addresses the problem of sparse and imbalanced label spaces in multi-task rPPG for healthcare monitoring, representing an incremental advancement with a novel method for a known bottleneck.
The paper tackles the challenge of achieving generalizability in multi-task remote physiological measurement (rPPG) for vital signs like heart rate, respiration, and blood oxygen saturation, and introduces PhysMLE, which improves performance through a mixture of low-rank experts and prior knowledge, as validated on a new benchmark and dataset.
Remote photoplethysmography (rPPG) has been widely applied to measure heart rate from face videos. To increase the generalizability of the algorithms, domain generalization (DG) attracted increasing attention in rPPG. However, when rPPG is extended to simultaneously measure more vital signs (e.g., respiration and blood oxygen saturation), achieving generalizability brings new challenges. Although partial features shared among different physiological signals can benefit multi-task learning, the sparse and imbalanced target label space brings the seesaw effect over task-specific feature learning. To resolve this problem, we designed an end-to-end Mixture of Low-rank Experts for multi-task remote Physiological measurement (PhysMLE), which is based on multiple low-rank experts with a novel router mechanism, thereby enabling the model to adeptly handle both specifications and correlations within tasks. Additionally, we introduced prior knowledge from physiology among tasks to overcome the imbalance of label space under real-world multi-task physiological measurement. For fair and comprehensive evaluations, this paper proposed a large-scale multi-task generalization benchmark, named Multi-Source Synsemantic Domain Generalization (MSSDG) protocol. Extensive experiments with MSSDG and intra-dataset have shown the effectiveness and efficiency of PhysMLE. In addition, a new dataset was collected and made publicly available to meet the needs of the MSSDG.