Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability
Provides theoretical guarantees for implicit regularization in deep linear networks, relevant for understanding generalization in overparameterized models.
This paper derives spectral conditions for low-rank implicit regularization in deep matrix factorization and proves that this low-rank phase persists under noise, with explicit bounds on perturbation effects.
This paper studies the stability of low-rank implicit regularization in perturbed deep matrix factorization, where the target matrix is corrupted by a noise matrix. We first derive sufficient spectral conditions under which gradient descent exhibits a low-rank phase in the noiseless setting. These conditions show how the target spectrum, initialization, and step size jointly determine the existence of a nonempty low-rank interval. We then analyze the perturbed gradient descent dynamics, proving convergence guarantees and quantifying how the perturbation affects iteration complexity and eigenvalue recovery. Finally, we show that the low-rank phase persists under perturbation, with explicit dependence on the perturbation size. Numerical experiments support the theoretical findings.