GAIA: Categorical Foundations of Generative AI
This work addresses foundational theoretical problems in machine learning for researchers, but it appears incremental as it builds on existing categorical methods without demonstrating practical gains.
The paper tackles the problem of providing a categorical foundation for generative AI by proposing GAIA, an architecture based on category theory and simplicial complexes, resulting in a coalgebraic formulation of deep learning that models backpropagation as an endofunctor.
In this paper, we propose GAIA, a generative AI architecture based on category theory. GAIA is based on a hierarchical model where modules are organized as a simplicial complex. Each simplicial complex updates its internal parameters biased on information it receives from its superior simplices and in turn relays updates to its subordinate sub-simplices. Parameter updates are formulated in terms of lifting diagrams over simplicial sets, where inner and outer horn extensions correspond to different types of learning problems. Backpropagation is modeled as an endofunctor over the category of parameters, leading to a coalgebraic formulation of deep learning.