Multi-person Implicit Reconstruction from a Single Image
This addresses the problem of realistic 3D capture of multiple interacting people for applications like VR/AR and animation, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of reconstructing detailed 3D models of multiple clothed people from a single image, addressing limitations of existing methods that struggle with loose clothing, hair, and occlusions. The result is a new end-to-end learning framework that achieves state-of-the-art performance with significant improvements in accuracy and completeness on both synthetic and real-world datasets.
We present a new end-to-end learning framework to obtain detailed and spatially coherent reconstructions of multiple people from a single image. Existing multi-person methods suffer from two main drawbacks: they are often model-based and therefore cannot capture accurate 3D models of people with loose clothing and hair; or they require manual intervention to resolve occlusions or interactions. Our method addresses both limitations by introducing the first end-to-end learning approach to perform model-free implicit reconstruction for realistic 3D capture of multiple clothed people in arbitrary poses (with occlusions) from a single image. Our network simultaneously estimates the 3D geometry of each person and their 6DOF spatial locations, to obtain a coherent multi-human reconstruction. In addition, we introduce a new synthetic dataset that depicts images with a varying number of inter-occluded humans and a variety of clothing and hair styles. We demonstrate robust, high-resolution reconstructions on images of multiple humans with complex occlusions, loose clothing and a large variety of poses and scenes. Our quantitative evaluation on both synthetic and real-world datasets demonstrates state-of-the-art performance with significant improvements in the accuracy and completeness of the reconstructions over competing approaches.