RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement
This work solves the problem of accurate and efficient non-contact physiological measurement for applications like healthcare and affective computing, representing an incremental improvement over existing deep learning methods.
The paper tackles the challenge of remote photoplethysmography (rPPG) by addressing the trade-off between capturing long-range dependencies and computational efficiency, achieving state-of-the-art performance with a 319% throughput increase and 23% peak GPU memory reduction.
Remote photoplethysmography (rPPG) is a method for non-contact measurement of physiological signals from facial videos, holding great potential in various applications such as healthcare, affective computing, and anti-spoofing. Existing deep learning methods struggle to address two core issues of rPPG simultaneously: understanding the periodic pattern of rPPG among long contexts and addressing large spatiotemporal redundancy in video segments. These represent a trade-off between computational complexity and the ability to capture long-range dependencies. In this paper, we introduce RhythmMamba, a state space model-based method that captures long-range dependencies while maintaining linear complexity. By viewing rPPG as a time series task through the proposed frame stem, the periodic variations in pulse waves are modeled as state transitions. Additionally, we design multi-temporal constraint and frequency domain feed-forward, both aligned with the characteristics of rPPG time series, to improve the learning capacity of Mamba for rPPG signals. Extensive experiments show that RhythmMamba achieves state-of-the-art performance with 319% throughput and 23% peak GPU memory. The codes are available at https://github.com/zizheng-guo/RhythmMamba.