DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images
This work addresses an under-explored intermediate case in computer vision for 3D reconstruction, reducing the image requirement from three or more to two, which is incremental but practical for applications with limited data.
The paper tackles the photometric stereo problem using only two differently illuminated images, proposing DeepPS2, a self-supervised deep learning framework that jointly estimates surface normals, albedo, and lighting without ground truth data, achieving competitive results on benchmark datasets.
Photometric stereo, a problem of recovering 3D surface normals using images of an object captured under different lightings, has been of great interest and importance in computer vision research. Despite the success of existing traditional and deep learning-based methods, it is still challenging due to: (i) the requirement of three or more differently illuminated images, (ii) the inability to model unknown general reflectance, and (iii) the requirement of accurate 3D ground truth surface normals and known lighting information for training. In this work, we attempt to address an under-explored problem of photometric stereo using just two differently illuminated images, referred to as the PS2 problem. It is an intermediate case between a single image-based reconstruction method like Shape from Shading (SfS) and the traditional Photometric Stereo (PS), which requires three or more images. We propose an inverse rendering-based deep learning framework, called DeepPS2, that jointly performs surface normal, albedo, lighting estimation, and image relighting in a completely self-supervised manner with no requirement of ground truth data. We demonstrate how image relighting in conjunction with image reconstruction enhances the lighting estimation in a self-supervised setting.