Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion
This work addresses the underdetermined problem of surface normal estimation for computer vision applications, representing an incremental improvement over existing deep learning methods by more efficiently utilizing inter-image information.
The paper tackled the problem of estimating surface normals from images under unknown lighting in uncalibrated photometric stereo by introducing an inter-intra image feature fusion module to guide per-image feature extraction and reduce ambiguity. The method achieved significantly better results than state-of-the-art methods on synthetic and real data, particularly improving performance on dark materials.
Uncalibrated photometric stereo is proposed to estimate the detailed surface normal from images under varying and unknown lightings. Recently, deep learning brings powerful data priors to this underdetermined problem. This paper presents a new method for deep uncalibrated photometric stereo, which efficiently utilizes the inter-image representation to guide the normal estimation. Previous methods use optimization-based neural inverse rendering or a single size-independent pooling layer to deal with multiple inputs, which are inefficient for utilizing information among input images. Given multi-images under different lighting, we consider the intra-image and inter-image variations highly correlated. Motivated by the correlated variations, we designed an inter-intra image feature fusion module to introduce the inter-image representation into the per-image feature extraction. The extra representation is used to guide the per-image feature extraction and eliminate the ambiguity in normal estimation. We demonstrate the effect of our design on a wide range of samples, especially on dark materials. Our method produces significantly better results than the state-of-the-art methods on both synthetic and real data.