DiffPoseNet: Direct Differentiable Camera Pose Estimation
This addresses robustness and cross-dataset generalization issues in camera pose estimation for applications like robotics and augmented reality, though it is incremental as it builds on direct methods.
The paper tackles camera pose estimation by introducing DiffPoseNet, which uses normal flow and a differentiable cheirality constraint to directly estimate 3D motion without relying on scene structure or optical flow, achieving competitive results on datasets like KITTI, TartanAir, and TUM-RGBD.
Current deep neural network approaches for camera pose estimation rely on scene structure for 3D motion estimation, but this decreases the robustness and thereby makes cross-dataset generalization difficult. In contrast, classical approaches to structure from motion estimate 3D motion utilizing optical flow and then compute depth. Their accuracy, however, depends strongly on the quality of the optical flow. To avoid this issue, direct methods have been proposed, which separate 3D motion from depth estimation but compute 3D motion using only image gradients in the form of normal flow. In this paper, we introduce a network NFlowNet, for normal flow estimation which is used to enforce robust and direct constraints. In particular, normal flow is used to estimate relative camera pose based on the cheirality (depth positivity) constraint. We achieve this by formulating the optimization problem as a differentiable cheirality layer, which allows for end-to-end learning of camera pose. We perform extensive qualitative and quantitative evaluation of the proposed DiffPoseNet's sensitivity to noise and its generalization across datasets. We compare our approach to existing state-of-the-art methods on KITTI, TartanAir, and TUM-RGBD datasets.