Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras
This addresses the challenge of accurate 3D human performance capture with fewer cameras, which is incremental over prior work.
The paper tackles the problem of estimating 3D body shape and pose from minimal camera views, achieving improved reconstruction accuracy and lower pose estimation error on datasets like Human 3.6M and TotalCapture.
We aim to simultaneously estimate the 3D articulated pose and high fidelity volumetric occupancy of human performance, from multiple viewpoint video (MVV) with as few as two views. We use a multi-channel symmetric 3D convolutional encoder-decoder with a dual loss to enforce the learning of a latent embedding that enables inference of skeletal joint positions and a volumetric reconstruction of the performance. The inference is regularised via a prior learned over a dataset of view-ablated multi-view video footage of a wide range of subjects and actions, and show this to generalise well across unseen subjects and actions. We demonstrate improved reconstruction accuracy and lower pose estimation error relative to prior work on two MVV performance capture datasets: Human 3.6M and TotalCapture.