Erzeugunsgrad, VC-Dimension and Neural Networks with rational activation function
This work provides a theoretical foundation linking algebraic geometry and learning theory, with applications to neural networks, but it is incremental as it builds on prior concepts.
The authors extended the notion of Erzeugungsgrad to connect affine Intersection Theory with VC-Theory, proving that VC-dimension and Krull dimension are linearly related up to logarithmic factors, and applied this to analyze neural networks with rational activation functions.
The notion of Erzeugungsgrad was introduced by Joos Heintz in 1983 to bound the number of non-empty cells occurring after a process of quantifier elimination. We extend this notion and the combinatorial bounds of Theorem 2 in Heintz (1983) using the degree for constructible sets defined in Pardo-Sebastián (2022). We show that the Erzeugungsgrad is the key ingredient to connect affine Intersection Theory over algebraically closed fields and the VC-Theory of Computational Learning Theory for families of classifiers given by parameterized families of constructible sets. In particular, we prove that the VC-dimension and the Krull dimension are linearly related up to logarithmic factors based on Intersection Theory. Using this relation, we study the density of correct test sequences in evasive varieties. We apply these ideas to analyze parameterized families of neural networks with rational activation function.