Conditional Prior Networks for Optical Flow
This work addresses the challenge of improving optical flow estimation for computer vision applications by developing a method that balances prior learning and generalization, though it is incremental in combining supervised and unsupervised elements.
The paper tackles the problem of learning richer priors for optical flow estimation by introducing Conditional Prior Networks (CPN), which capture long-range correlations and semantics from annotated data without overfitting. The result shows that CPN, when combined with a simple optical flow architecture, outperforms variational and unsupervised methods and performs comparably to fully supervised ones while being more transferable across datasets.
Classical computation of optical flow involves generic priors (regularizers) that capture rudimentary statistics of images, but not long-range correlations or semantics. On the other hand, fully supervised methods learn the regularity in the annotated data, without explicit regularization and with the risk of overfitting. We seek to learn richer priors on the set of possible flows that are statistically compatible with an image. Once the prior is learned in a supervised fashion, one can easily learn the full map to infer optical flow directly from two or more images, without any need for (additional) supervision. We introduce a novel architecture, called Conditional Prior Network (CPN), and show how to train it to yield a conditional prior. When used in conjunction with a simple optical flow architecture, the CPN beats all variational methods and all unsupervised learning-based ones using the same data term. It performs comparably to fully supervised ones, that however are fine-tuned to a particular dataset. Our method, on the other hand, performs well even when transferred between datasets.