Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization
This addresses the problem of more accurate scene understanding for computer vision applications, though it is incremental as it builds on existing unsupervised approaches.
The paper tackles joint optical flow and camera motion estimation in rigid scenes by incorporating epipolar geometry into an unsupervised deep learning framework, resulting in improved optical flow quality and better camera motion estimates compared to other unsupervised methods.
We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.