Radio Frequency Signal based Human Silhouette Segmentation: A Sequential Diffusion Approach
This work addresses the problem of accurate human silhouette segmentation in complex environments for applications like surveillance or human-computer interaction, representing an incremental improvement over existing methods.
The paper tackles human silhouette segmentation using radio frequency signals by proposing a two-stage Sequential Diffusion Model that progressively synthesizes segmentation maps while incorporating motion dynamics, achieving state-of-the-art performance with an IoU of 0.732 on the HIBER benchmark.
Radio frequency (RF) signals have been proved to be flexible for human silhouette segmentation (HSS) under complex environments. Existing studies are mainly based on a one-shot approach, which lacks a coherent projection ability from the RF domain. Additionally, the spatio-temporal patterns have not been fully explored for human motion dynamics in HSS. Therefore, we propose a two-stage Sequential Diffusion Model (SDM) to progressively synthesize high-quality segmentation jointly with the considerations on motion dynamics. Cross-view transformation blocks are devised to guide the diffusion model in a multi-scale manner for comprehensively characterizing human related patterns in an individual frame such as directional projection from signal planes. Moreover, spatio-temporal blocks are devised to fine-tune the frame-level model to incorporate spatio-temporal contexts and motion dynamics, enhancing the consistency of the segmentation maps. Comprehensive experiments on a public benchmark -- HIBER demonstrate the state-of-the-art performance of our method with an IoU 0.732. Our code is available at https://github.com/ph-w2000/SDM.