FNIN: A Fourier Neural Operator-based Numerical Integration Network for Surface-form-gradients
This addresses the challenge of high-accuracy, high-resolution surface reconstruction for applications like photometric stereo, though it is an incremental improvement over existing deep learning approaches.
The paper tackles the problem of reconstructing 3D surfaces from gradients (SfG) by proposing FNIN, a Fourier neural operator-based network, which achieves errors of fewer than 0.1 mm on tested objects and improves accuracy and efficiency over state-of-the-art methods.
Surface-from-gradients (SfG) aims to recover a three-dimensional (3D) surface from its gradients. Traditional methods encounter significant challenges in achieving high accuracy and handling high-resolution inputs, particularly facing the complex nature of discontinuities and the inefficiencies associated with large-scale linear solvers. Although recent advances in deep learning, such as photometric stereo, have enhanced normal estimation accuracy, they do not fully address the intricacies of gradient-based surface reconstruction. To overcome these limitations, we propose a Fourier neural operator-based Numerical Integration Network (FNIN) within a two-stage optimization framework. In the first stage, our approach employs an iterative architecture for numerical integration, harnessing an advanced Fourier neural operator to approximate the solution operator in Fourier space. Additionally, a self-learning attention mechanism is incorporated to effectively detect and handle discontinuities. In the second stage, we refine the surface reconstruction by formulating a weighted least squares problem, addressing the identified discontinuities rationally. Extensive experiments demonstrate that our method achieves significant improvements in both accuracy and efficiency compared to current state-of-the-art solvers. This is particularly evident in handling high-resolution images with complex data, achieving errors of fewer than 0.1 mm on tested objects.