Escaping Saddle Points in Ill-Conditioned Matrix Completion with a Scalable Second Order Method
This addresses a critical bottleneck in matrix completion for applications requiring high accuracy with limited data, though it is incremental as it builds on existing IRLS and non-convex optimization methods.
The paper tackles the problem of low-rank matrix completion for ill-conditioned matrices by proposing an iterative algorithm that combines IRLS and saddle-escaping smoothing Newton methods, achieving local quadratic convergence near the information limit and successfully completing matrices with condition numbers up to 10^10 from few samples.
We propose an iterative algorithm for low-rank matrix completion that can be interpreted as both an iteratively reweighted least squares (IRLS) algorithm and a saddle-escaping smoothing Newton method applied to a non-convex rank surrogate objective. It combines the favorable data efficiency of previous IRLS approaches with an improved scalability by several orders of magnitude. Our method attains a local quadratic convergence rate already for a number of samples that is close to the information theoretical limit. We show in numerical experiments that unlike many state-of-the-art approaches, our approach is able to complete very ill-conditioned matrices with a condition number of up to $10^{10}$ from few samples.