Polarimetric Multi-View Inverse Rendering
This addresses 3D reconstruction challenges for computer vision applications by integrating polarimetric cues, offering an incremental improvement over existing methods.
The paper tackles 3D reconstruction by proposing Polarimetric Multi-View Inverse Rendering, which uses multi-view color polarization images to refine an initial 3D model by optimizing photometric and polarimetric rendering errors, resulting in detailed shape reconstruction without material-specific assumptions as demonstrated on synthetic and real data.
A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) of reflected light is related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric and polarimetric rendering errors using multi-view RGB and AoP images, where we propose a novel polarimetric rendering cost function that enables us to effectively constrain each estimated surface vertex's normal while considering four possible ambiguous azimuth angles revealed from the AoP measurement. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific polarized reflection depending on the material.