Physics of Learning: A Lagrangian perspective to different learning paradigms
This work proposes a foundational physics-inspired perspective for unifying different learning paradigms, potentially impacting all of ML/AI by offering a new theoretical lens.
The paper tackles the problem of building efficient learning systems that minimize the number of observations to reach a desired error threshold, by deriving classic learning algorithms like Bellman's optimality equation and the Adam optimizer from a unified Lagrangian framework based on least action principles.
We study the problem of building an efficient learning system. Efficient learning processes information in the least time, i.e., building a system that reaches a desired error threshold with the least number of observations. Building upon least action principles from physics, we derive classic learning algorithms, Bellman's optimality equation in reinforcement learning, and the Adam optimizer in generative models from first principles, i.e., the Learning $\textit{Lagrangian}$. We postulate that learning searches for stationary paths in the Lagrangian, and learning algorithms are derivable by seeking the stationary trajectories.