UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss
This addresses the challenge of domain mismatch in optical flow for computer vision applications, offering a solution for scenarios with scarce ground truth data.
The paper tackled the problem of optical flow estimation without labeled data by proposing an unsupervised loss based on occlusion-aware bidirectional flow and census transform, achieving state-of-the-art unsupervised results on KITTI benchmarks and competitive accuracy with supervised methods.
In the era of end-to-end deep learning, many advances in computer vision are driven by large amounts of labeled data. In the optical flow setting, however, obtaining dense per-pixel ground truth for real scenes is difficult and thus such data is rare. Therefore, recent end-to-end convolutional networks for optical flow rely on synthetic datasets for supervision, but the domain mismatch between training and test scenarios continues to be a challenge. Inspired by classical energy-based optical flow methods, we design an unsupervised loss based on occlusion-aware bidirectional flow estimation and the robust census transform to circumvent the need for ground truth flow. On the KITTI benchmarks, our unsupervised approach outperforms previous unsupervised deep networks by a large margin, and is even more accurate than similar supervised methods trained on synthetic datasets alone. By optionally fine-tuning on the KITTI training data, our method achieves competitive optical flow accuracy on the KITTI 2012 and 2015 benchmarks, thus in addition enabling generic pre-training of supervised networks for datasets with limited amounts of ground truth.