3D Reconstruction of Novel Object Shapes from Single Images
This addresses a key challenge in computer vision for applications like robotics and AR/VR, but it appears incremental as it builds on prior learning-based approaches.
The paper tackles the problem of 3D shape reconstruction from single images, focusing on unseen objects, and shows that their SDFNet achieves state-of-the-art performance compared to existing methods like GenRe and OccNet.
Accurately predicting the 3D shape of any arbitrary object in any pose from a single image is a key goal of computer vision research. This is challenging as it requires a model to learn a representation that can infer both the visible and occluded portions of any object using a limited training set. A training set that covers all possible object shapes is inherently infeasible. Such learning-based approaches are inherently vulnerable to overfitting, and successfully implementing them is a function of both the architecture design and the training approach. We present an extensive investigation of factors specific to architecture design, training, experiment design, and evaluation that influence reconstruction performance and measurement. We show that our proposed SDFNet achieves state-of-the-art performance on seen and unseen shapes relative to existing methods GenRe and OccNet. We provide the first large-scale evaluation of single image shape reconstruction to unseen objects. The source code, data and trained models can be found on https://github.com/rehg-lab/3DShapeGen.