Local and global topological complexity measures OF ReLU neural network functions
This work addresses theoretical understanding of neural network complexity for researchers in machine learning and topology, but it appears incremental as it builds on existing Morse theory frameworks.
The paper tackles the problem of quantifying topological complexity in ReLU neural networks by applying generalized piecewise-linear Morse theory to define new local and global measures, showing that local complexity can be arbitrarily high.
We apply a generalized piecewise-linear (PL) version of Morse theory due to Grunert-Kuhnel-Rote to define and study new local and global notions of topological complexity for fully-connected feedforward ReLU neural network functions, F: R^n -> R. Along the way, we show how to construct, for each such F, a canonical polytopal complex K(F) and a deformation retract of the domain onto K(F), yielding a convenient compact model for performing calculations. We also give a construction showing that local complexity can be arbitrarily high.