DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks
This work addresses the need for accurate heart and breathing rate monitoring in health care and human-computer interaction, enabling robust measurements in practical scenarios with major motions.
The authors tackled the problem of non-contact video-based physiological measurement under challenging conditions like head rotations and heterogeneous lighting, achieving significant performance improvements over all current state-of-the-art methods on RGB and infrared video datasets.
Non-contact video-based physiological measurement has many applications in health care and human-computer interaction. Practical applications require measurements to be accurate even in the presence of large head rotations. We propose the first end-to-end system for video-based measurement of heart and breathing rate using a deep convolutional network. The system features a new motion representation based on a skin reflection model and a new attention mechanism using appearance information to guide motion estimation, both of which enable robust measurement under heterogeneous lighting and major motions. Our approach significantly outperforms all current state-of-the-art methods on both RGB and infrared video datasets. Furthermore, it allows spatial-temporal distributions of physiological signals to be visualized via the attention mechanism.