FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases
This addresses the challenge of resource-intensive optical flow training for researchers and practitioners with limited hardware access.
The paper tackles the problem of optical flow estimation by developing FlowSeek, a framework that requires minimal hardware resources for training. It achieves superior cross-dataset generalization with relative improvements of 10% and 15% over the previous state-of-the-art on Sintel Final and KITTI datasets.
We present FlowSeek, a novel framework for optical flow requiring minimal hardware resources for training. FlowSeek marries the latest advances on the design space of optical flow networks with cutting-edge single-image depth foundation models and classical low-dimensional motion parametrization, implementing a compact, yet accurate architecture. FlowSeek is trained on a single consumer-grade GPU, a hardware budget about 8x lower compared to most recent methods, and still achieves superior cross-dataset generalization on Sintel Final and KITTI, with a relative improvement of 10 and 15% over the previous state-of-the-art SEA-RAFT, as well as on Spring and LayeredFlow datasets.