TASI Lectures on Physics for Machine Learning
This provides a foundational educational resource for researchers and students in machine learning and theoretical physics, but it is incremental as it synthesizes existing knowledge.
The lectures present an overview of neural network theory from a field-theoretic physics perspective, covering classic results like universal approximation and recent developments such as neural tangent kernels and Kolmogorov-Arnold networks.
These notes are based on lectures I gave at TASI 2024 on Physics for Machine Learning. The focus is on neural network theory, organized according to network expressivity, statistics, and dynamics. I present classic results such as the universal approximation theorem and neural network / Gaussian process correspondence, and also more recent results such as the neural tangent kernel, feature learning with the maximal update parameterization, and Kolmogorov-Arnold networks. The exposition on neural network theory emphasizes a field theoretic perspective familiar to theoretical physicists. I elaborate on connections between the two, including a neural network approach to field theory.