On the Geometry of Deep Learning
This provides a theoretical framework for researchers to better understand and enhance deep learning models, though it is incremental as it builds on existing work over the past decade.
The paper tackles the problem of understanding deep learning's mathematical foundations by exploring the connection between deep networks and affine spline approximations, focusing on how these networks tessellate input space to analyze and improve their inner workings.
In this paper, we overview one promising avenue of progress at the mathematical foundation of deep learning: the connection between deep networks and function approximation by affine splines (continuous piecewise linear functions in multiple dimensions). In particular, we will overview work over the past decade on understanding certain geometrical properties of a deep network's affine spline mapping, in particular how it tessellates its input space. As we will see, the affine spline connection and geometrical viewpoint provide a powerful portal through which to view, analyze, and improve the inner workings of a deep network.