NEAT: Distilling 3D Wireframes from Neural Attraction Fields
It addresses structured 3D reconstruction for computer vision applications, offering a novel approach that improves efficiency and accuracy over prior methods.
This paper tackles 3D wireframe reconstruction by proposing NEAT, a method that uses neural fields and bipartite matching to distill 3D line segments and junctions from 2D observations, achieving state-of-the-art results on DTU and BlendedMVS datasets and providing a better initialization for 3D Gaussian Splatting with about 20 times fewer initial points.
This paper studies the problem of structured 3D reconstruction using wireframes that consist of line segments and junctions, focusing on the computation of structured boundary geometries of scenes. Instead of leveraging matching-based solutions from 2D wireframes (or line segments) for 3D wireframe reconstruction as done in prior arts, we present NEAT, a rendering-distilling formulation using neural fields to represent 3D line segments with 2D observations, and bipartite matching for perceiving and distilling of a sparse set of 3D global junctions. The proposed {NEAT} enjoys the joint optimization of the neural fields and the global junctions from scratch, using view-dependent 2D observations without precomputed cross-view feature matching. Comprehensive experiments on the DTU and BlendedMVS datasets demonstrate our NEAT's superiority over state-of-the-art alternatives for 3D wireframe reconstruction. Moreover, the distilled 3D global junctions by NEAT, are a better initialization than SfM points, for the recently-emerged 3D Gaussian Splatting for high-fidelity novel view synthesis using about 20 times fewer initial 3D points. Project page: \url{https://xuenan.net/neat}.