On the Topology of Neural Network Superlevel Sets
This provides theoretical guarantees on the topological properties of neural network outputs, which is foundational for understanding their behavior in applications like dynamical systems and optimization.
The paper demonstrates that neural networks with specific activation functions produce outputs with controlled topological complexity, establishing architecture-only bounds on Betti numbers for superlevel sets and Lie bracket rank drop loci uniformly across all weight parameters.
We show that neural networks with activations satisfying a Riccati-type ordinary differential equation condition, an assumption arising in recent universal approximation results in the uniform topology, produce Pfaffian outputs on analytic domains with format controlled only by the architecture. Consequently, superlevel sets, as well as Lie bracket rank drop loci for neural network parameterized vector fields, admit architecture-only bounds on topological complexity, in particular on total Betti numbers, uniformly over all weights.