Progressive Batching for Efficient Non-linear Least Squares
This addresses computational bottlenecks in model fitting for computer vision and related fields, though it appears incremental by combining existing ideas from stochastic optimization and statistics.
The paper tackles the computational inefficiency of non-linear least squares solvers by introducing a progressive batching method that guarantees convergence and reduces computation. Empirical results show competitive convergence rates on computer vision tasks like image alignment and essential matrix estimation with large numbers of residuals.
Non-linear least squares solvers are used across a broad range of offline and real-time model fitting problems. Most improvements of the basic Gauss-Newton algorithm tackle convergence guarantees or leverage the sparsity of the underlying problem structure for computational speedup. With the success of deep learning methods leveraging large datasets, stochastic optimization methods received recently a lot of attention. Our work borrows ideas from both stochastic machine learning and statistics, and we present an approach for non-linear least-squares that guarantees convergence while at the same time significantly reduces the required amount of computation. Empirical results show that our proposed method achieves competitive convergence rates compared to traditional second-order approaches on common computer vision problems, such as image alignment and essential matrix estimation, with very large numbers of residuals.