NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics
This work addresses a gap in polarized inverse rendering by extending support to conductors, which are common in real-world scenes, though it appears incremental as it builds on existing polarization-based methods.
The paper tackles the problem of inverse rendering for both conductors and dielectrics, which previous methods only supported for dielectrics, leading to errors with conductors due to their different reflection properties and strong specular reflections; it proposes NeISF++ with a general pBRDF and a geometry initialization method using DoLP images, achieving superior results in geometry and material decomposition and downstream tasks like relighting on synthetic and real datasets.
Recent inverse rendering methods have greatly improved shape, material, and illumination reconstruction by utilizing polarization cues. However, existing methods only support dielectrics, ignoring conductors that are found everywhere in life. Since conductors and dielectrics have different reflection properties, using previous conductor methods will lead to obvious errors. In addition, conductors are glossy, which may cause strong specular reflection and is hard to reconstruct. To solve the above issues, we propose NeISF++, an inverse rendering pipeline that supports conductors and dielectrics. The key ingredient for our proposal is a general pBRDF that describes both conductors and dielectrics. As for the strong specular reflection problem, we propose a novel geometry initialization method using DoLP images. This physical cue is invariant to intensities and thus robust to strong specular reflections. Experimental results on our synthetic and real datasets show that our method surpasses the existing polarized inverse rendering methods for geometry and material decomposition as well as downstream tasks like relighting.