Implicit Mesh Reconstruction from Unannotated Image Collections
This addresses the challenge of 3D reconstruction with limited supervision for computer vision applications, though it is incremental over prior work.
The paper tackles the problem of inferring 3D shape, texture, and camera pose from a single RGB image using only category-level image collections with foreground masks as supervision, and it shows competitive performance compared to methods with stronger supervision.
We present an approach to infer the 3D shape, texture, and camera pose for an object from a single RGB image, using only category-level image collections with foreground masks as supervision. We represent the shape as an image-conditioned implicit function that transforms the surface of a sphere to that of the predicted mesh, while additionally predicting the corresponding texture. To derive supervisory signal for learning, we enforce that: a) our predictions when rendered should explain the available image evidence, and b) the inferred 3D structure should be geometrically consistent with learned pixel to surface mappings. We empirically show that our approach improves over prior work that leverages similar supervision, and in fact performs competitively to methods that use stronger supervision. Finally, as our method enables learning with limited supervision, we qualitatively demonstrate its applicability over a set of about 30 object categories.