CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces
This addresses the challenge of accurate 3D reconstruction for complex surfaces in computer vision, representing an incremental improvement over existing photometric stereo techniques.
The paper tackled the problem of photometric stereo for non-convex surfaces, where conventional methods fail due to global light transport, by introducing a CNN-based network that learns relationships from input images to surface normals, achieving superior performance on highly non-convex scenes compared to traditional algorithms.
Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called {\it observation map} that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.