Algebraic Dynamical Systems in Machine Learning
This work provides a foundational framework for unifying and extending dynamic models in machine learning, potentially impacting researchers in AI and ML theory.
The authors introduced an algebraic analogue of dynamical systems using term rewriting, showing that it can embed major dynamic machine learning architectures like recurrent neural networks and diffusion models, and proposed it as a template for generalizing these models to structured or non-numerical data.
We introduce an algebraic analogue of dynamical systems, based on term rewriting. We show that a recursive function applied to the output of an iterated rewriting system defines a formal class of models into which all the main architectures for dynamic machine learning models (including recurrent neural networks, graph neural networks, and diffusion models) can be embedded. Considered in category theory, we also show that these algebraic models are a natural language for describing the compositionality of dynamic models. Furthermore, we propose that these models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data, including 'hybrid symbolic-numeric' models.