On implicit regularization: Morse functions and applications to matrix factorization
This work addresses the fundamental problem of explaining generalization in deep models like neural networks, providing theoretical insights that could benefit researchers in machine learning theory.
The authors tackled the problem of understanding implicit regularization in deep learning by developing a new criterion based on dynamical systems and Morse functions, which they applied to prove a conjecture about implicit regularization in matrix factorization.
In this paper, we revisit implicit regularization from the ground up using notions from dynamical systems and invariant subspaces of Morse functions. The key contributions are a new criterion for implicit regularization---a leading contender to explain the generalization power of deep models such as neural networks---and a general blueprint to study it. We apply these techniques to settle a conjecture on implicit regularization in matrix factorization.