DITTO: Dual and Integrated Latent Topologies for Implicit 3D Reconstruction
This addresses the challenge of accurate 3D reconstruction for applications like computer vision and robotics, representing an incremental improvement by combining existing latent types.
The paper tackles the problem of implicit 3D reconstruction from noisy and sparse point clouds by proposing DITTO, which uses dual point and grid latents to enhance stability and detail, resulting in high-fidelity reconstructions that surpass previous state-of-the-art methods on object- and scene-level datasets.
We propose a novel concept of dual and integrated latent topologies (DITTO in short) for implicit 3D reconstruction from noisy and sparse point clouds. Most existing methods predominantly focus on single latent type, such as point or grid latents. In contrast, the proposed DITTO leverages both point and grid latents (i.e., dual latent) to enhance their strengths, the stability of grid latents and the detail-rich capability of point latents. Concretely, DITTO consists of dual latent encoder and integrated implicit decoder. In the dual latent encoder, a dual latent layer, which is the key module block composing the encoder, refines both latents in parallel, maintaining their distinct shapes and enabling recursive interaction. Notably, a newly proposed dynamic sparse point transformer within the dual latent layer effectively refines point latents. Then, the integrated implicit decoder systematically combines these refined latents, achieving high-fidelity 3D reconstruction and surpassing previous state-of-the-art methods on object- and scene-level datasets, especially in thin and detailed structures.