Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry
This addresses a foundational problem in geometric computer vision for applications using consumer-grade cameras, though it is incremental as it builds on existing multi-perspective camera models.
The paper tackles the problem of relative pose estimation for rolling shutter cameras, which is crucial for dynamic computer vision applications like visual SLAM, by introducing a rolling shutter essential matrix and generalizing epipolar geometry concepts, achieving validation on dedicated benchmarks.
The vast majority of modern consumer-grade cameras employ a rolling shutter mechanism. In dynamic geometric computer vision applications such as visual SLAM, the so-called rolling shutter effect therefore needs to be properly taken into account. A dedicated relative pose solver appears to be the first problem to solve, as it is of eminent importance to bootstrap any derivation of multi-view geometry. However, despite its significance, it has received inadequate attention to date. This paper presents a detailed investigation of the geometry of the rolling shutter relative pose problem. We introduce the rolling shutter essential matrix, and establish its link to existing models such as the push-broom cameras, summarized in a clean hierarchy of multi-perspective cameras. The generalization of well-established concepts from epipolar geometry is completed by a definition of the Sampson distance in the rolling shutter case. The work is concluded with a careful investigation of the introduced epipolar geometry for rolling shutter cameras on several dedicated benchmarks.