Refractive Light-Field Features for Curved Transparent Objects in Structure from Motion
This addresses the challenge of robots operating around refractive objects like glass and acrylic, with applications in manufacturing and domestic tasks, though it is incremental as it builds on existing light-field and structure-from-motion techniques.
The paper tackles the problem of robotic vision failing on curved transparent objects by proposing a novel light-field image feature that detects refracted light patterns, resulting in improved structure-from-motion performance with 15-35% more frequent convergence than state-of-the-art methods.
Curved refractive objects are common in the human environment, and have a complex visual appearance that can cause robotic vision algorithms to fail. Light-field cameras allow us to address this challenge by capturing the view-dependent appearance of such objects in a single exposure. We propose a novel image feature for light fields that detects and describes the patterns of light refracted through curved transparent objects. We derive characteristic points based on these features allowing them to be used in place of conventional 2D features. Using our features, we demonstrate improved structure-from-motion performance in challenging scenes containing refractive objects, including quantitative evaluations that show improved camera pose estimates and 3D reconstructions. Additionally, our methods converge 15-35% more frequently than the state-of-the-art. Our method is a critical step towards allowing robots to operate around refractive objects, with applications in manufacturing, quality assurance, pick-and-place, and domestic robots working with acrylic, glass and other transparent materials.