Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images
This addresses the challenge of 3D reconstruction for computer vision applications without requiring labeled data, though it is incremental as it builds on prior unsupervised and symmetry-based approaches.
The paper tackles the problem of inferring 3D shape from single-view images without supervision, achieving this by using a generative autoencoding model that exploits object symmetry, and it demonstrates superior accuracy on benchmarks compared to supervised methods.
We show that generative models can be used to capture visual geometry constraints statistically. We use this fact to infer the 3D shape of object categories from raw single-view images. Differently from prior work, we use no external supervision, nor do we use multiple views or videos of the objects. We achieve this by a simple reconstruction task, exploiting the symmetry of the objects' shape and albedo. Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder. Our experiments show that this method can recover the 3D shape of human faces, cat faces, and cars from single view images, without supervision. On benchmarks, we demonstrate superior accuracy compared to other methods that use supervision at the level of 2D image correspondences.