Preconditioners for the Stochastic Training of Neural Fields
This work addresses the training efficiency problem for researchers and practitioners using neural fields in computer vision and robotics, but it is incremental as it builds on existing optimization methods.
The paper tackled the slow training times of neural fields by exploring alternative optimization techniques, proposing curvature-aware diagonal preconditioners that demonstrated effectiveness in tasks like image reconstruction and NeRF without sacrificing accuracy.
Neural fields encode continuous multidimensional signals as neural networks, enabling diverse applications in computer vision, robotics, and geometry. While Adam is effective for stochastic optimization, it often requires long training times. To address this, we explore alternative optimization techniques to accelerate training without sacrificing accuracy. Traditional second-order methods like L-BFGS are unsuitable for stochastic settings. We propose a theoretical framework for training neural fields with curvature-aware diagonal preconditioners, demonstrating their effectiveness across tasks such as image reconstruction, shape modeling, and Neural Radiance Fields (NeRF).