Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
This addresses the challenge of 3D reconstruction from single images for computer vision applications, offering a more efficient annotation approach, though it appears incremental in method.
The paper tackles the problem of reconstructing 3D shape and pose from a single image by proposing a method that uses cheaper 2D silhouette annotations and re-projects predicted shapes back onto the image, demonstrating superiority in evaluation on object categories.
An emerging problem in computer vision is the reconstruction of 3D shape and pose of an object from a single image. Hitherto, the problem has been addressed through the application of canonical deep learning methods to regress from the image directly to the 3D shape and pose labels. These approaches, however, are problematic from two perspectives. First, they are minimizing the error between 3D shapes and pose labels - with little thought about the nature of this label error when reprojecting the shape back onto the image. Second, they rely on the onerous and ill-posed task of hand labeling natural images with respect to 3D shape and pose. In this paper we define the new task of pose-aware shape reconstruction from a single image, and we advocate that cheaper 2D annotations of objects silhouettes in natural images can be utilized. We design architectures of pose-aware shape reconstruction which re-project the predicted shape back on to the image using the predicted pose. Our evaluation on several object categories demonstrates the superiority of our method for predicting pose-aware 3D shapes from natural images.