DiFT: Differentiable Differential Feature Transform for Multi-View Stereo
This addresses the need for improved feature extraction in 3D reconstruction for computer vision applications, though it appears incremental as it builds on existing multi-view stereo methods.
The paper tackles the problem of generating discriminative and view-invariant features for multi-view stereo by learning to transform differential cues from images, resulting in enhanced 3D reconstruction that compares favorably with state-of-the-art techniques on challenging objects.
We present a novel framework to automatically learn to transform the differential cues from a stack of images densely captured with a rotational motion into spatially discriminative and view-invariant per-pixel features at each view. These low-level features can be directly fed to any existing multi-view stereo technique for enhanced 3D reconstruction. The lighting condition during acquisition can also be jointly optimized in a differentiable fashion. We sample from a dozen of pre-scanned objects with a wide variety of geometry and reflectance to synthesize a large amount of high-quality training data. The effectiveness of our features is demonstrated on a number of challenging objects acquired with a lightstage, comparing favorably with state-of-the-art techniques. Finally, we explore additional applications of geometric detail visualization and computational stylization of complex appearance.