MetaFi: Device-Free Pose Estimation via Commodity WiFi for Metaverse Avatar Simulation
This addresses privacy and illumination issues in smart homes for metaverse applications, though it is incremental as it adapts existing WiFi sensing techniques to a new domain.
The paper tackles human pose estimation for metaverse avatars by proposing MetaFi, a device-free method using commodity WiFi, which achieves a PCK@50 of 95.23% in real-world experiments.
Avatar refers to a representative of a physical user in the virtual world that can engage in different activities and interact with other objects in metaverse. Simulating the avatar requires accurate human pose estimation. Though camera-based solutions yield remarkable performance, they encounter the privacy issue and degraded performance caused by varying illumination, especially in smart home. In this paper, we propose a WiFi-based IoT-enabled human pose estimation scheme for metaverse avatar simulation, namely MetaFi. Specifically, a deep neural network is designed with customized convolutional layers and residual blocks to map the channel state information to human pose landmarks. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. WiFi is ubiquitous and robust to illumination, making it a feasible solution for avatar applications in smart home. The experiments are conducted in the real world, and the results show that the MetaFi achieves very high performance with a PCK@50 of 95.23%.