Learning Photometric Feature Transform for Free-form Object Scan
This addresses the challenge of accurate 3D scanning for free-form objects with varying illumination, using a lightweight setup like a camera with LEDs or a tablet, though it appears incremental as it builds on multi-view stereo methods.
The paper tackles the problem of 3D reconstruction from hand-held scans by learning to transform photometric measurements into view-invariant features, which are then used in a multi-view stereo pipeline to enhance reconstruction of geometry and anisotropic reflectance. The results compare favorably with state-of-the-art techniques and are validated against professional 3D scanner data.
We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amount of synthetic data. We further build a system to reconstruct both the geometry and anisotropic reflectance of a variety of challenging objects from hand-held scans. The effectiveness of the system is demonstrated with a lightweight prototype, consisting of a camera and an array of LEDs, as well as an off-the-shelf tablet. Our results are validated against reconstructions from a professional 3D scanner and photographs, and compare favorably with state-of-the-art techniques.