Cross-Spectral Neural Radiance Fields
This addresses the challenge of integrating multi-spectral and infrared data for scene modeling, which is incremental as it builds on Neural Radiance Fields.
The paper tackles the problem of learning cross-spectral scene representations from images captured by cameras with different light spectrum sensitivities, proposing X-NeRF to render aligned images of different modalities from arbitrary viewpoints, with experiments on 16 forward-facing scenes confirming its effectiveness.
We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity, based on the Neural Radiance Fields formulation. X-NeRF optimizes camera poses across spectra during training and exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints, which are aligned and at the same resolution. Experiments on 16 forward-facing scenes, featuring color, multi-spectral and infrared images, confirm the effectiveness of X-NeRF at modeling Cross-Spectral scene representations.