MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask
This solves the occlusion issue in optical flow estimation for computer vision applications, representing a strong incremental improvement.
The paper tackles the problem of optical flow estimation by addressing ambiguity from occluded areas during feature warping, proposing MaskFlownet which learns an occlusion mask without explicit supervision and achieves state-of-the-art performance on MPI Sintel, KITTI 2012, and KITTI 2015 benchmarks.
Feature warping is a core technique in optical flow estimation; however, the ambiguity caused by occluded areas during warping is a major problem that remains unsolved. In this paper, we propose an asymmetric occlusion-aware feature matching module, which can learn a rough occlusion mask that filters useless (occluded) areas immediately after feature warping without any explicit supervision. The proposed module can be easily integrated into end-to-end network architectures and enjoys performance gains while introducing negligible computational cost. The learned occlusion mask can be further fed into a subsequent network cascade with dual feature pyramids with which we achieve state-of-the-art performance. At the time of submission, our method, called MaskFlownet, surpasses all published optical flow methods on the MPI Sintel, KITTI 2012 and 2015 benchmarks. Code is available at https://github.com/microsoft/MaskFlownet.