AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training
This addresses the challenge of high-fidelity 3D scene reconstruction for computer vision and graphics applications, representing an incremental improvement over existing NeRF methods.
The paper tackles the problem of training Neural Radiance Fields (NeRFs) with high-resolution data to overcome limitations like large parameters, misaligned inputs, and smooth details, achieving recovery of more high-frequency details compared to state-of-the-art NeRF models.
Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function. Though NeRF is able to render complex 3D scenes with view-dependent effects, few efforts have been devoted to exploring its limits in a high-resolution setting. Specifically, existing NeRF-based methods face several limitations when reconstructing high-resolution real scenes, including a very large number of parameters, misaligned input data, and overly smooth details. In this work, we conduct the first pilot study on training NeRF with high-resolution data and propose the corresponding solutions: 1) marrying the multilayer perceptron (MLP) with convolutional layers which can encode more neighborhood information while reducing the total number of parameters; 2) a novel training strategy to address misalignment caused by moving objects or small camera calibration errors; and 3) a high-frequency aware loss. Our approach is nearly free without introducing obvious training/testing costs, while experiments on different datasets demonstrate that it can recover more high-frequency details compared with the current state-of-the-art NeRF models. Project page: \url{https://yifanjiang.net/alignerf.}