Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild
This addresses robust 3D reconstruction from single images in the wild for computer vision applications, but it is incremental as it builds on existing zero-shot methods.
The paper tackles the problem of monocular 3D shape reconstruction failing in real-world conditions due to imperfect segmentation and occlusions, achieving state-of-the-art zero-shot results on real-world images with fewer parameters.
Recent monocular 3D shape reconstruction methods have shown promising zero-shot results on object-segmented images without any occlusions. However, their effectiveness is significantly compromised in real-world conditions, due to imperfect object segmentation by off-the-shelf models and the prevalence of occlusions. To effectively address these issues, we propose a unified regression model that integrates segmentation and reconstruction, specifically designed for occlusion-aware 3D shape reconstruction. To facilitate its reconstruction in the wild, we also introduce a scalable data synthesis pipeline that simulates a wide range of variations in objects, occluders, and backgrounds. Training on our synthetic data enables the proposed model to achieve state-of-the-art zero-shot results on real-world images, using significantly fewer parameters than competing approaches.