Ambient-robust Inverse Rendering using Active RGB-NIR Imaging
For computer vision and graphics researchers, this method addresses the problem of ambient illumination sensitivity in inverse rendering, offering a practical solution for robust reconstruction under varying lighting conditions.
This work introduces an ambient-robust inverse rendering method using active RGB-NIR imaging that leverages NIR flash illumination to achieve stable point-light shading invariant to ambient light. The method outperforms prior approaches in accurate geometry and reflectance estimation across multiple ambient lighting scenarios.
Inverse rendering aims to reconstruct geometry and reflectance of objects from images. Despite recent progress, existing methods often produces inaccurate reconstructions that are sensitive to ambient illumination conditions. Here we introduce an ambient-robust inverse rendering method enabled by active RGB-NIR imaging. Our key insight is to leverage near-infrared (NIR) flash illumination-imperceptible to human observers-to obtain stable point-light shading that is largely invariant to ambient illumination. By using multi-view RGB images illuminated by ambient light and NIR images acquired with active NIR flash illumination, we reconstruct accurate geometry and reflectance by exploiting the complementary benefits of RGB and NIR images via a three-stage inverse rendering method. To enable dense multi-view acquisition, we develop an active imaging system equipped with a RGB-NIR camera and a NIR flash mounted on a mobile base. Using this system, we collect the first multi-view RGB-NIR inverse rendering dataset captured under multiple ambient illumination conditions. Experiments demonstrate that our method outperforms prior approaches, achieving accurate geometry and reflectance estimation across multiple ambient lighting scenarios.