Predictive online optimisation with applications to optical flow
This work addresses dynamic inverse problems in video processing, such as optical flow, with an incremental improvement to existing online optimization methods.
The paper tackles the problem of online optimization for dynamic inverse problems like video processing with optical flow by introducing a predictive online primal-dual proximal splitting method, resulting in excellent real-time performance in computational image stabilization and convergence in regularization theory.
Online optimisation revolves around new data being introduced into a problem while it is still being solved; think of deep learning as more training samples become available. We adapt the idea to dynamic inverse problems such as video processing with optical flow. We introduce a corresponding predictive online primal-dual proximal splitting method. The video frames now exactly correspond to the algorithm iterations. A user-prescribed predictor describes the evolution of the primal variable. To prove convergence we need a predictor for the dual variable based on (proximal) gradient flow. This affects the model that the method asymptotically minimises. We show that for inverse problems the effect is, essentially, to construct a new dynamic regulariser based on infimal convolution of the static regularisers with the temporal coupling. We finish by demonstrating excellent real-time performance of our method in computational image stabilisation and convergence in terms of regularisation theory.