D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning
This work addresses a challenging task in computer vision with applications in robotics and graphics, but it is incremental as it builds directly on existing Occupancy Networks.
The paper tackles the problem of 3D reconstruction from a single 2D image by extending Occupancy Networks with cross-domain learning between image and point cloud domains, resulting in improved visual quality and detail capture compared to the baseline.
Deep learning based 3D reconstruction of single view 2D image is becoming increasingly popular due to their wide range of real-world applications, but this task is inherently challenging because of the partial observability of an object from a single perspective. Recently, state of the art probability based Occupancy Networks reconstructed 3D surfaces from three different types of input domains: single view 2D image, point cloud and voxel. In this study, we extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Specifically, we first convert the single view 2D image into a simpler point cloud representation, and then reconstruct a 3D surface from it. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.