Learning Energy Based Inpainting for Optical Flow
This work addresses the need for more interpretable and modular optical flow methods in computer vision, though it appears incremental as it builds on existing inpainting-based approaches.
The authors tackled the problem of creating an interpretable and lightweight optical flow method by proposing a three-step inpainting-based algorithm with an optimization layer for efficient backpropagation, achieving competitive performance with state-of-the-art networks.
Modern optical flow methods are often composed of a cascade of many independent steps or formulated as a black box neural network that is hard to interpret and analyze. In this work we seek for a plain, interpretable, but learnable solution. We propose a novel inpainting based algorithm that approaches the problem in three steps: feature selection and matching, selection of supporting points and energy based inpainting. To facilitate the inference we propose an optimization layer that allows to backpropagate through 10K iterations of a first-order method without any numerical or memory problems. Compared to recent state-of-the-art networks, our modular CNN is very lightweight and competitive with other, more involved, inpainting based methods.