Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models
This work addresses camera-based vital sign monitoring for applications like driver-monitoring and healthcare, representing an incremental improvement by integrating existing techniques in a novel way.
The paper tackled the problem of recovering pulse waveforms from facial video for imaging photoplethysmography by combining signal processing and deep learning in an inverse problem framework, achieving state-of-the-art heart rate estimation performance with less than one-fifth the parameters of competing methods.
Camera-based monitoring of vital signs, also known as imaging photoplethysmography (iPPG), has seen applications in driver-monitoring, perfusion assessment in surgical settings, affective computing, and more. iPPG involves sensing the underlying cardiac pulse from video of the skin and estimating vital signs such as the heart rate or a full pulse waveform. Some previous iPPG methods impose model-based sparse priors on the pulse signals and use iterative optimization for pulse wave recovery, while others use end-to-end black-box deep learning methods. In contrast, we introduce methods that combine signal processing and deep learning methods in an inverse problem framework. Our methods estimate the underlying pulse signal and heart rate from facial video by learning deep-network-based denoising operators that leverage deep algorithm unfolding and deep equilibrium models. Experiments show that our methods can denoise an acquired signal from the face and infer the correct underlying pulse rate, achieving state-of-the-art heart rate estimation performance on well-known benchmarks, all with less than one-fifth the number of learnable parameters as the closest competing method.