ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation
This addresses the problem of expensive 3D annotations in computer vision for researchers and practitioners, though it is incremental as it builds on self-supervised and transformer-based approaches.
The paper tackles unsupervised viewpoint estimation by formulating it as a self-supervised learning task using image reconstruction, where pairs of images with unknown viewpoints provide supervision. It demonstrates that a perspective spatial transformer enables efficient learning, outperforming existing unsupervised methods on synthetic data and achieving competitive results on PASCAL3D+.
Understanding the 3D world without supervision is currently a major challenge in computer vision as the annotations required to supervise deep networks for tasks in this domain are expensive to obtain on a large scale. In this paper, we address the problem of unsupervised viewpoint estimation. We formulate this as a self-supervised learning task, where image reconstruction provides the supervision needed to predict the camera viewpoint. Specifically, we make use of pairs of images of the same object at training time, from unknown viewpoints, to self-supervise training by combining the viewpoint information from one image with the appearance information from the other. We demonstrate that using a perspective spatial transformer allows efficient viewpoint learning, outperforming existing unsupervised approaches on synthetic data, and obtains competitive results on the challenging PASCAL3D+ dataset.