Momentum Point-Perplexity Mechanics in Large Language Models
This work addresses the challenge of making large language models more predictable and aligned, offering potential applications in interpretability and control, though it is incremental in applying existing physics concepts to AI.
The paper tackled the problem of understanding and controlling internal hidden state dynamics in large language models by applying a physics-based approach, finding that a conserved 'energy' quantity emerges and enabling a control method called Jacobian steering that improved semantic quality in continuations.
We take a physics-based approach to studying how the internal hidden states of large language models change from token to token during inference. Across 20 open-source transformer models (135M-3B parameters), we find that a quantity combining the rate of change in hidden states and the model's next-token certainty, analogous to energy in physics, remains nearly constant. Random-weight models conserve this "energy" more tightly than pre-trained ones, while training shifts models into a faster, more decisive regime with greater variability. Using this "log-Lagrangian" view, we derive a control method called Jacobian steering, which perturbs hidden states in the minimal way needed to favor a target token. This approach maintained near-constant energy in two tested models and produced continuations rated higher in semantic quality than the models' natural outputs. Viewing transformers through this mechanics lens offers a principled basis for interpretability, anomaly detection, and low-risk steering. This could help make powerful models more predictable and aligned with human intent.