FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments
This provides an expandable and efficient solution for multi-person motion capture in applications like sports or surveillance, though it is incremental as it builds on existing sensor fusion techniques.
The paper tackles the problem of accurately capturing global multi-person motions in open environments without calibration, achieving significant performance improvements over state-of-the-art single-modal methods on Human-M3 and FreeMotion datasets.
We propose a novel hybrid calibration-free method FreeCap to accurately capture global multi-person motions in open environments. Our system combines a single LiDAR with expandable moving cameras, allowing for flexible and precise motion estimation in a unified world coordinate. In particular, We introduce a local-to-global pose-aware cross-sensor human-matching module that predicts the alignment among each sensor, even in the absence of calibration. Additionally, our coarse-to-fine sensor-expandable pose optimizer further optimizes the 3D human key points and the alignments, it is also capable of incorporating additional cameras to enhance accuracy. Extensive experiments on Human-M3 and FreeMotion datasets demonstrate that our method significantly outperforms state-of-the-art single-modal methods, offering an expandable and efficient solution for multi-person motion capture across various applications.