Bulk-boundary decomposition of neural networks
This provides a theoretical tool for understanding deep learning training, though it appears incremental as it builds on existing gradient descent formulations.
The paper introduces the bulk-boundary decomposition framework to analyze neural network training dynamics, showing it separates data-independent intrinsic dynamics from data-dependent stochastic interactions, and extends this to a field-theoretic formulation.
We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a natural extension, we develop a field-theoretic formulation of neural dynamics based on this decomposition.