Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos
This work addresses the open problem of explaining deep neural networks theoretically for researchers in machine learning and dynamical systems, but it appears incremental as it applies existing dynamical systems concepts to DNNs without claiming major breakthroughs.
The paper tackles the theoretical understanding of deep neural networks by modeling them as discrete-time dynamical systems and analyzing their complexity through topological entropy and chaos via Lyapunov exponents, with results applied to classification and generalization capabilities.
The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapunov exponents. The properties revealed for the dynamics of DNN are applied to analyze the corresponding capabilities of classification and generalization.