BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurements
This addresses the need for efficient and unified physiological monitoring, though it is incremental as it builds on existing multi-task and temporal shift methods.
The paper tackles the problem of joint camera-based measurement of facial actions, cardiac, and pulmonary signals by proposing BigSmall, an efficient multi-task architecture that reduces computational costs while achieving comparable or better accuracy than task-specific models.
Understanding of human visual perception has historically inspired the design of computer vision architectures. As an example, perception occurs at different scales both spatially and temporally, suggesting that the extraction of salient visual information may be made more effective by paying attention to specific features at varying scales. Visual changes in the body due to physiological processes also occur at different scales and with modality-specific characteristic properties. Inspired by this, we present BigSmall, an efficient architecture for physiological and behavioral measurement. We present the first joint camera-based facial action, cardiac, and pulmonary measurement model. We propose a multi-branch network with wrapping temporal shift modules that yields both accuracy and efficiency gains. We observe that fusing low-level features leads to suboptimal performance, but that fusing high level features enables efficiency gains with negligible loss in accuracy. Experimental results demonstrate that BigSmall significantly reduces the computational costs. Furthermore, compared to existing task-specific models, BigSmall achieves comparable or better results on multiple physiological measurement tasks simultaneously with a unified model.