Deep Implicit Moving Least-Squares Functions for 3D Reconstruction
This work addresses the challenge of generating high-quality 3D shapes from point sets for applications in computer graphics and vision, representing an incremental advancement by integrating a known surface formulation with deep learning.
The paper tackles the problem of representing continuous and fine geometry from discrete point sets in 3D deep learning by introducing implicit moving least-squares (IMLS) surfaces into neural networks, resulting in improved reconstruction quality and computational efficiency compared to state-of-the-art methods.
Point set is a flexible and lightweight representation widely used for 3D deep learning. However, their discrete nature prevents them from representing continuous and fine geometry, posing a major issue for learning-based shape generation. In this work, we turn the discrete point sets into smooth surfaces by introducing the well-known implicit moving least-squares (IMLS) surface formulation, which naturally defines locally implicit functions on point sets. We incorporate IMLS surface generation into deep neural networks for inheriting both the flexibility of point sets and the high quality of implicit surfaces. Our IMLSNet predicts an octree structure as a scaffold for generating MLS points where needed and characterizes shape geometry with learned local priors. Furthermore, our implicit function evaluation is independent of the neural network once the MLS points are predicted, thus enabling fast runtime evaluation. Our experiments on 3D object reconstruction demonstrate that IMLSNets outperform state-of-the-art learning-based methods in terms of reconstruction quality and computational efficiency. Extensive ablation tests also validate our network design and loss functions.