Understanding over-parameterized deep networks by geometrization
This work addresses a foundational problem in AI by providing a geometric framework for deep learning, though it appears incremental as it builds on existing geometric ideas.
The authors tackled the problem of understanding over-parameterized deep networks by proposing a geometrization scheme, showing that Riemannian geometry explains properties like generalization and convergence, and noting similarities to quantum computation systems.
A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.