360° Optical Flow using Tangent Images
This work addresses a domain-specific challenge in computer vision and robotics for processing omnidirectional images, representing an incremental improvement over existing methods.
The paper tackled the problem of estimating optical flow in 360° images by proposing a method based on tangent images to mitigate distortions from equirectangular projection, resulting in improved flow estimation as demonstrated quantitatively and qualitatively in experiments.
Omnidirectional 360° images have found many promising and exciting applications in computer vision, robotics and other fields, thanks to their increasing affordability, portability and their 360° field of view. The most common format for storing, processing and visualising 360° images is equirectangular projection (ERP). However, the distortion introduced by the nonlinear mapping from 360° image to ERP image is still a barrier that holds back ERP images from being used as easily as conventional perspective images. This is especially relevant when estimating 360° optical flow, as the distortions need to be mitigated appropriately. In this paper, we propose a 360° optical flow method based on tangent images. Our method leverages gnomonic projection to locally convert ERP images to perspective images, and uniformly samples the ERP image by projection to a cubemap and regular icosahedron vertices, to incrementally refine the estimated 360° flow fields even in the presence of large rotations. Our experiments demonstrate the benefits of our proposed method both quantitatively and qualitatively.