Intrinsic Image Fusion for Multi-View 3D Material Reconstruction
This work addresses material reconstruction for computer graphics and vision applications, offering an incremental improvement by better constraining optimization with priors and fusion techniques.
The paper tackles the underconstrained problem of reconstructing physically based materials from multi-view images by introducing Intrinsic Image Fusion, which incorporates single-view priors and a robust optimization framework to fuse inconsistent predictions, resulting in outperforming state-of-the-art methods in material disentanglement on synthetic and real scenes with sharp and clean reconstructions suitable for high-quality relighting.
We introduce Intrinsic Image Fusion, a method that reconstructs high-quality physically based materials from multi-view images. Material reconstruction is highly underconstrained and typically relies on analysis-by-synthesis, which requires expensive and noisy path tracing. To better constrain the optimization, we incorporate single-view priors into the reconstruction process. We leverage a diffusion-based material estimator that produces multiple, but often inconsistent, candidate decompositions per view. To reduce the inconsistency, we fit an explicit low-dimensional parametric function to the predictions. We then propose a robust optimization framework using soft per-view prediction selection together with confidence-based soft multi-view inlier set to fuse the most consistent predictions of the most confident views into a consistent parametric material space. Finally, we use inverse path tracing to optimize for the low-dimensional parameters. Our results outperform state-of-the-art methods in material disentanglement on both synthetic and real scenes, producing sharp and clean reconstructions suitable for high-quality relighting.