Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior
This work addresses the challenge of 3D object reconstruction for semantic scene understanding by introducing uncertainty modeling, which is incremental as it builds on existing neural shape prior methods.
The paper tackles the problem of reconstructing detailed 3D shapes from monocular images by modeling uncertainty in the reconstruction process, resulting in improved accuracy through uncertainty-based fusion in synthetic and real datasets.
3D object reconstruction is important for semantic scene understanding. It is challenging to reconstruct detailed 3D shapes from monocular images directly due to a lack of depth information, occlusion and noise. Most current methods generate deterministic object models without any awareness of the uncertainty of the reconstruction. We tackle this problem by leveraging a neural object representation which learns an object shape distribution from large dataset of 3d object models and maps it into a latent space. We propose a method to model uncertainty as part of the representation and define an uncertainty-aware encoder which generates latent codes with uncertainty directly from individual input images. Further, we propose a method to propagate the uncertainty in the latent code to SDF values and generate a 3d object mesh with local uncertainty for each mesh component. Finally, we propose an incremental fusion method under a Bayesian framework to fuse the latent codes from multi-view observations. We evaluate the system in both synthetic and real datasets to demonstrate the effectiveness of uncertainty-based fusion to improve 3D object reconstruction accuracy.