Thermodynamic Irreversibility of Training Algorithms
For AI researchers studying learning dynamics, this work provides a foundational theoretical understanding of irreversibility in training algorithms, though it is primarily theoretical and does not present experimental validation.
The paper establishes a framework for defining and analyzing the irreversibility of training algorithms in AI, showing that four distinct characterizations of irreversibility are equivalent to leading order in step size. It reveals that irreversibility breaks non-isometric continuous reparametrization symmetries while preserving orthogonal symmetries, leading to a universal preference for learning trajectories that minimize entropy production rate.
The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towards understanding the learning dynamics of modern AI systems. In this work, we establish a general framework for defining and analyzing the irreversibility of training algorithms. We show that four different ways to characterize the irreversibility of dynamical processes are equivalent to leading order in the step size $η$: numerical backward error $ϕ_{\rm DE}$, time-renormalized correction $ϕ_{\rm TR}$, microscopic time reversal asymmetry $ϕ_{\rm TA}$, and the (regularized) stochastic-thermodynamic entropy production $ϕ_{\rm ST}$. The irreversibility gives rise to a time-reversal-symmetry-breaking emergent force that generically breaks non-isometric continuous reparametrization symmetries, preserves orthogonal symmetries, and leads to a universal preference for those learning trajectories that minimize the entropy production rate.