FAWN: Floor-And-Walls Normal Regularization for Direct Neural TSDF Reconstruction
This work addresses 3D reconstruction quality for applications in robotics or AR/VR, but it is incremental as it modifies existing TSDF methods.
The paper tackles the problem of distorted room shapes and local artifacts in direct 3D reconstruction by proposing FAWN, a method that uses floor and wall normal regularization to correct these issues, resulting in quality gains across multiple benchmarks like SCANNET and ICL-NUIM.
Leveraging 3D semantics for direct 3D reconstruction has a great potential yet unleashed. For instance, by assuming that walls are vertical, and a floor is planar and horizontal, we can correct distorted room shapes and eliminate local artifacts such as holes, pits, and hills. In this paper, we propose FAWN, a modification of truncated signed distance function (TSDF) reconstruction methods, which considers scene structure by detecting walls and floor in a scene, and penalizing the corresponding surface normals for deviating from the horizontal and vertical directions. Implemented as a 3D sparse convolutional module, FAWN can be incorporated into any trainable pipeline that predicts TSDF. Since FAWN requires 3D semantics only for training, no additional limitations on further use are imposed. We demonstrate, that FAWN-modified methods use semantics more effectively, than existing semantic-based approaches. Besides, we apply our modification to state-of-the-art TSDF reconstruction methods, and demonstrate a quality gain in SCANNET, ICL-NUIM, TUM RGB-D, and 7SCENES benchmarks.