Detail Preserving Residual Feature Pyramid Modules for Optical Flow
This work addresses detail preservation in optical flow estimation for computer vision applications, offering an incremental improvement through a modular enhancement to existing iterative refinement architectures.
The paper tackles the problem of detail loss in optical flow estimation caused by downsampling in feature pyramids, which blends foreground and background objects, by proposing a Residual Feature Pyramid Module (RFPM) that retains details without altering iterative refinement designs. Results show RFPM visibly reduces flow errors, improves state-of-the-art performance on Sintel clean pass, and ranks among top methods on KITTI, with a transfer learning approach that dramatically decreases training time.
Feature pyramids and iterative refinement have recently led to great progress in optical flow estimation. However, downsampling in feature pyramids can cause blending of foreground objects with the background, which will mislead subsequent decisions in the iterative processing. The results are missing details especially in the flow of thin and of small structures. We propose a novel Residual Feature Pyramid Module (RFPM) which retains important details in the feature map without changing the overall iterative refinement design of the optical flow estimation. RFPM incorporates a residual structure between multiple feature pyramids into a downsampling module that corrects the blending of objects across boundaries. We demonstrate how to integrate our module with two state-of-the-art iterative refinement architectures. Results show that our RFPM visibly reduces flow errors and improves state-of-art performance in the clean pass of Sintel, and is one of the top-performing methods in KITTI. According to the particular modular structure of RFPM, we introduce a special transfer learning approach that can dramatically decrease the training time compared to a typical full optical flow training schedule on multiple datasets.