LiFF: Light Field Features in Scale and Depth
This addresses a problem for computer vision researchers and practitioners in tasks like structure from motion, offering a novel method for robust feature matching across viewpoints.
The paper tackles the problem of feature detection and description failing under challenging light transport effects like occlusion and reflective surfaces by introducing LiFF, a computationally efficient 4D light field feature detector and descriptor. It demonstrates significantly improved 3D reconstructions via structure from motion compared to leading methods and runs an order of magnitude faster than the leading 4D approach.
Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on. Unfortunately these fail in the presence of challenging light transport effects including partial occlusion, low contrast, and reflective or refractive surfaces. Building on spatio-angular imaging modalities offered by emerging light field cameras, we introduce a new and computationally efficient 4D light field feature detector and descriptor: LiFF. LiFF is scale invariant and utilizes the full 4D light field to detect features that are robust to changes in perspective. This is particularly useful for structure from motion (SfM) and other tasks that match features across viewpoints of a scene. We demonstrate significantly improved 3D reconstructions via SfM when using LiFF instead of the leading 2D or 4D features, and show that LiFF runs an order of magnitude faster than the leading 4D approach. Finally, LiFF inherently estimates depth for each feature, opening a path for future research in light field-based SfM.