Unsupervised Learning of Depth Estimation and Visual Odometry for Sparse Light Field Cameras
This work addresses the problem of integrating diverse imaging devices in robotics, offering an incremental improvement for autonomous camera interpretation.
The paper tackles the challenge of interpreting sparse light field cameras for robotics by generalizing unsupervised learning techniques to estimate depth and visual odometry, achieving more accurate results with metric scale compared to monocular and conventional 4D imagery methods.
While an exciting diversity of new imaging devices is emerging that could dramatically improve robotic perception, the challenges of calibrating and interpreting these cameras have limited their uptake in the robotics community. In this work we generalise techniques from unsupervised learning to allow a robot to autonomously interpret new kinds of cameras. We consider emerging sparse light field (LF) cameras, which capture a subset of the 4D LF function describing the set of light rays passing through a plane. We introduce a generalised encoding of sparse LFs that allows unsupervised learning of odometry and depth. We demonstrate the proposed approach outperforming monocular and conventional techniques for dealing with 4D imagery, yielding more accurate odometry and depth maps and delivering these with metric scale. We anticipate our technique to generalise to a broad class of LF and sparse LF cameras, and to enable unsupervised recalibration for coping with shifts in camera behaviour over the lifetime of a robot. This work represents a first step toward streamlining the integration of new kinds of imaging devices in robotics applications.