Les Houches Lectures on Deep Learning at Large & Infinite Width
This work provides foundational insights for researchers in machine learning and statistical physics, though it is incremental as it synthesizes existing theoretical concepts.
The lectures addressed the theoretical properties of deep neural networks in the infinite-width limit and large-width regime, covering statistical and dynamical aspects such as connections to linear models, kernels, and Gaussian processes, with results including perturbative and non-perturbative analyses.
These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training.