An Application of the Holonomic Gradient Method to the Neural Tangent Kernel
This work addresses a computational bottleneck in neural network theory for researchers in machine learning, but appears incremental as it applies existing mathematical tools to a specific context.
The paper tackles the problem of numerically evaluating dual activations for neural tangent kernels using holonomic distributions, and presents methods based on computer algebra algorithms for differential operators.
A holonomic system of linear partial differential equations is, roughly speaking, a system whose solution space is finite dimensional. A distribution that is a solution of a holonomic system is called a holonomic distribution. We give methods to numerically evaluate dual activations of holonomic activator distributions for neural tangent kernels. These methods are based on computer algebra algorithms for rings of differential operators.