FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras
This addresses the problem of non-intrusive motion capture for applications like animation or sports analysis, representing a novel integration of UAVs with computer vision techniques.
The paper tackles markerless motion capture of moving characters in wide spaces by using three autonomous flying cameras to track and fuse depth data, achieving competent and plausible surface and motion reconstruction results.
Aiming at automatic, convenient and non-instrusive motion capture, this paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras (autonomous unmanned aerial vehicles(UAV) each integrated with an RGBD video camera). During data capture, three cooperative flying cameras automatically track and follow the moving target who performs large scale motions in a wide space. We propose a novel non-rigid surface registration method to track and fuse the depth of the three flying cameras for surface motion tracking of the moving target, and simultaneously calculate the pose of each flying camera. We leverage the using of visual-odometry information provided by the UAV platform, and formulate the surface tracking problem in a non-linear objective function that can be linearized and effectively minimized through a Gaussian-Newton method. Quantitative and qualitative experimental results demonstrate the competent and plausible surface and motion reconstruction results