Categories of Differentiable Polynomial Circuits for Machine Learning
This work addresses a foundational gap in the semantic study of machine learning algorithms by focusing on model classes, which is incremental as it builds on existing reverse derivative category theory.
The paper tackles the problem of defining and characterizing model classes for machine learning within the framework of reverse derivative categories, proposing polynomial circuits as a suitable model and providing an axiomatization and functional completeness proof. It also explores applications of these circuits over specific semirings for discrete-valued machine learning.
Reverse derivative categories (RDCs) have recently been shown to be a suitable semantic framework for studying machine learning algorithms. Whereas emphasis has been put on training methodologies, less attention has been devoted to particular \emph{model classes}: the concrete categories whose morphisms represent machine learning models. In this paper we study presentations by generators and equations of classes of RDCs. In particular, we propose \emph{polynomial circuits} as a suitable machine learning model. We give an axiomatisation for these circuits and prove a functional completeness result. Finally, we discuss the use of polynomial circuits over specific semirings to perform machine learning with discrete values.