How do language models learn facts? Dynamics, curricula and hallucinations
This work addresses the problem of understanding knowledge acquisition dynamics in language models for AI researchers, offering insights into training efficiency and hallucination mitigation, though it is incremental in nature.
The study investigated how language models learn factual knowledge using a synthetic recall task, finding that learning occurs in three phases with a performance plateau, and that imbalanced data distributions shorten this plateau while hallucinations arise alongside knowledge acquisition.
Large language models accumulate vast knowledge during pre-training, yet the dynamics governing this acquisition remain poorly understood. This work investigates the learning dynamics of language models on a synthetic factual recall task, uncovering three key findings: First, language models learn in three phases, exhibiting a performance plateau before acquiring precise factual knowledge. Mechanistically, this plateau coincides with the formation of attention-based circuits that support recall. Second, the training data distribution significantly impacts learning dynamics, as imbalanced distributions lead to shorter plateaus. Finally, hallucinations emerge simultaneously with knowledge, and integrating new knowledge into the model through fine-tuning is challenging, as it quickly corrupts its existing parametric memories. Our results emphasize the importance of data distribution in knowledge acquisition and suggest novel data scheduling strategies to accelerate neural network training.