Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry
It proposes a foundational research direction bridging algebraic geometry and deep learning, potentially impacting theoretical understanding but is incremental as it builds on existing ideas.
The paper promotes studying machine learning model function spaces using algebraic geometry, focusing on neural networks with polynomial activations to connect algebro-geometric invariants like dimension and degree to aspects such as sample complexity and expressivity.
In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic models, such as neural networks with polynomial activations, whose associated function spaces are semi-algebraic varieties. We outline a dictionary between algebro-geometric invariants of these varieties, such as dimension, degree, and singularities, and fundamental aspects of machine learning, such as sample complexity, expressivity, training dynamics, and implicit bias. Along the way, we review the literature and discuss ideas beyond the algebraic domain. This work lays the foundations of a research direction bridging algebraic geometry and deep learning, that we refer to as neuroalgebraic geometry.