MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior
This addresses the problem of efficient and accurate optical flow estimation for compressed videos, which is incremental as it builds on existing deep learning methods by incorporating pre-computed motion vectors.
The paper tackles optical flow estimation for compressed videos by leveraging motion vectors from compression streams, resulting in a model that reduces AEPE by 1.09 or saves 52% time compared to existing models.
In recent years, many deep learning-based methods have been proposed to tackle the problem of optical flow estimation and achieved promising results. However, they hardly consider that most videos are compressed and thus ignore the pre-computed information in compressed video streams. Motion vectors, one of the compression information, record the motion of the video frames. They can be directly extracted from the compression code stream without computational cost and serve as a solid prior for optical flow estimation. Therefore, we propose an optical flow model, MVFlow, which uses motion vectors to improve the speed and accuracy of optical flow estimation for compressed videos. In detail, MVFlow includes a key Motion-Vector Converting Module, which ensures that the motion vectors can be transformed into the same domain of optical flow and then be utilized fully by the flow estimation module. Meanwhile, we construct four optical flow datasets for compressed videos containing frames and motion vectors in pairs. The experimental results demonstrate the superiority of our proposed MVFlow, which can reduce the AEPE by 1.09 compared to existing models or save 52% time to achieve similar accuracy to existing models.