Spec-NeRF: Multi-spectral Neural Radiance Fields
This work addresses the challenge of multispectral scene reconstruction for applications in imaging and computer vision, offering a more practical alternative to specialized equipment, though it is incremental as it builds on existing NeRF methods.
The paper tackles the problem of reconstructing multispectral radiance fields and camera spectral sensitivity functions from filtered color images, achieving high fidelity and promising spectral reconstruction while retaining geometric understanding capabilities.
We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. The proposed method focuses on modeling the physical imaging process, and applies the estimated SSFs and radiance field to synthesize novel views of multispectral scenes. In this method, the data acquisition requires only a low-cost trichromatic camera and several off-the-shelf color filters, making it more practical than using specialized 3D scanning and spectral imaging equipment. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction while retaining the inherent capability of NeRF to comprehend geometric structures. Code is available at https://github.com/CPREgroup/SpecNeRF-v2.