A Quantum Field Theory of Representation Learning
This work provides a novel theoretical framework for improving training efficiency in time series models, though it is incremental as it builds on prior published research.
The paper tackles the problem of slow convergence in temporal representation learning by applying quantum field theory concepts, specifically gauge invariance, to loss functions with continuous symmetries, resulting in faster convergence.
Continuous symmetries and their breaking play a prominent role in contemporary physics. Effective low-energy field theories around symmetry breaking states explain diverse phenomena such as superconductivity, magnetism, and the mass of nucleons. We show that such field theories can also be a useful tool in machine learning, in particular for loss functions with continuous symmetries that are spontaneously broken by random initializations. In this paper, we illuminate our earlier published work (Bamler & Mandt, 2018) on this topic more from the perspective of theoretical physics. We show that the analogies between superconductivity and symmetry breaking in temporal representation learning are rather deep, allowing us to formulate a gauge theory of `charged' embedding vectors in time series models. We show that making the loss function gauge invariant speeds up convergence in such models.