Learning to Optimize Non-Rigid Tracking
This work addresses tracking robustness and speed for non-rigid motion in computer vision, representing an incremental improvement over existing nested-loop methods.
The paper tackles non-rigid tracking by integrating deep features into the tracking objective and using a learned preconditioner to speed up solver convergence, resulting in robust tracking on large motions and faster convergence than the original method by a large margin.
One of the widespread solutions for non-rigid tracking has a nested-loop structure: with Gauss-Newton to minimize a tracking objective in the outer loop, and Preconditioned Conjugate Gradient (PCG) to solve a sparse linear system in the inner loop. In this paper, we employ learnable optimizations to improve tracking robustness and speed up solver convergence. First, we upgrade the tracking objective by integrating an alignment data term on deep features which are learned end-to-end through CNN. The new tracking objective can capture the global deformation which helps Gauss-Newton to jump over local minimum, leading to robust tracking on large non-rigid motions. Second, we bridge the gap between the preconditioning technique and learning method by introducing a ConditionNet which is trained to generate a preconditioner such that PCG can converge within a small number of steps. Experimental results indicate that the proposed learning method converges faster than the original PCG by a large margin.