RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network
This addresses a specific problem in computer vision for 3D reconstruction, offering incremental improvements in handling difficult regions through feature fusion and attention mechanisms.
The paper tackles the challenge of predicting accurate normal maps from 2D images in complex regions with varying geometry and materials in photometric stereo, proposing RMAFF-PSN which outperforms most existing calibrated methods on benchmark datasets, especially for highly non-convex structures and sparse lighting conditions.
Predicting accurate normal maps of objects from two-dimensional images in regions of complex structure and spatial material variations is challenging using photometric stereo methods due to the influence of surface reflection properties caused by variations in object geometry and surface materials. To address this issue, we propose a photometric stereo network called a RMAFF-PSN that uses residual multiscale attentional feature fusion to handle the ``difficult'' regions of the object. Unlike previous approaches that only use stacked convolutional layers to extract deep features from the input image, our method integrates feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions, through shallow-deep stage feature extraction, double branching enhancement, and attention optimization. To test the network structure under real-world conditions, we propose a new real dataset called Simple PS data, which contains multiple objects with varying structures and materials. Experimental results on a publicly available benchmark dataset demonstrate that our method outperforms most existing calibrated photometric stereo methods for the same number of input images, especially in the case of highly non-convex object structures. Our method also obtains good results under sparse lighting conditions.