auto-fpt: Automating Free Probability Theory Calculations for Machine Learning Theory
This tool aims to streamline and automate tedious calculations for researchers in machine learning theory, facilitating the reproduction of known results and discovery of new phenomena, though it is incremental as it builds on existing free probability theory methods.
The authors tackled the problem of automating complex calculations in machine learning theory involving high-dimensional expected traces of random matrices, by introducing auto-fpt, a Python tool that automatically generates reduced fixed-point equations for symbolic computation using free probability theory.
A large part of modern machine learning theory often involves computing the high-dimensional expected trace of a rational expression of large rectangular random matrices. To symbolically compute such quantities using free probability theory, we introduce auto-fpt, a lightweight Python and SymPy-based tool that can automatically produce a reduced system of fixed-point equations which can be solved for the quantities of interest, and effectively constitutes a theory. We overview the algorithmic ideas underlying auto-fpt and its applications to various interesting problems, such as the high-dimensional error of linearized feed-forward neural networks, recovering well-known results. We hope that auto-fpt streamlines the majority of calculations involved in high-dimensional analysis, while helping the machine learning community reproduce known and uncover new phenomena.