Joint Epipolar Tracking (JET): Simultaneous optimization of epipolar geometry and feature correspondences
This work addresses pose estimation for applications like autonomous driving by introducing a novel optimization approach, though it appears incremental as it builds on existing direct methods.
The paper tackles the problem of pose estimation by proposing a sparse direct method that simultaneously optimizes epipolar geometry and feature correspondences using image intensity values, and demonstrates that it outperforms classical reprojection error optimization on synthetic and KITTI datasets while running in real-time on a single CPU thread.
Traditionally, pose estimation is considered as a two step problem. First, feature correspondences are determined by direct comparison of image patches, or by associating feature descriptors. In a second step, the relative pose and the coordinates of corresponding points are estimated, most often by minimizing the reprojection error (RPE). RPE optimization is based on a loss function that is merely aware of the feature pixel positions but not of the underlying image intensities. In this paper, we propose a sparse direct method which introduces a loss function that allows to simultaneously optimize the unscaled relative pose, as well as the set of feature correspondences directly considering the image intensity values. Furthermore, we show how to integrate statistical prior information on the motion into the optimization process. This constructive inclusion of a Bayesian bias term is particularly efficient in application cases with a strongly predictable (short term) dynamic, e.g. in a driving scenario. In our experiments, we demonstrate that the JET algorithm we propose outperforms the classical reprojection error optimization on two synthetic datasets and on the KITTI dataset. The JET algorithm runs in real-time on a single CPU thread.