4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras
This addresses the problem of efficient and robust motion capture in crowded scenes for applications like sports analysis or surveillance, representing a novel integration rather than an incremental improvement.
The paper tackles realtime multi-person motion capture from multiple video cameras by proposing a 4D association graph that unifies per-view parsing, cross-view matching, and temporal tracking into a single optimization framework, achieving 30fps with 5 cameras on a 5-person scene and outperforming state-of-the-art methods quantitatively.
This paper contributes a novel realtime multi-person motion capture algorithm using multiview video inputs. Due to the heavy occlusions in each view, joint optimization on the multiview images and multiple temporal frames is indispensable, which brings up the essential challenge of realtime efficiency. To this end, for the first time, we unify per-view parsing, cross-view matching, and temporal tracking into a single optimization framework, i.e., a 4D association graph that each dimension (image space, viewpoint and time) can be treated equally and simultaneously. To solve the 4D association graph efficiently, we further contribute the idea of 4D limb bundle parsing based on heuristic searching, followed with limb bundle assembling by proposing a bundle Kruskal's algorithm. Our method enables a realtime online motion capture system running at 30fps using 5 cameras on a 5-person scene. Benefiting from the unified parsing, matching and tracking constraints, our method is robust to noisy detection, and achieves high-quality online pose reconstruction quality. The proposed method outperforms the state-of-the-art method quantitatively without using high-level appearance information. We also contribute a multiview video dataset synchronized with a marker-based motion capture system for scientific evaluation.