Instability in Downstream Task Performance During LLM Pretraining
This addresses a practical issue for researchers and practitioners in selecting optimal checkpoints during LLM training, but it is incremental as it builds on existing checkpointing practices.
The study tackled the problem of fluctuating downstream task performance during LLM pretraining, which complicates checkpoint selection, and found that checkpoint averaging and ensemble methods improve stability without altering training.
When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest validation score. However, downstream metrics often exhibit substantial fluctuations, making it difficult to identify the checkpoint that truly represents the best-performing model. In this study, we empirically analyze the stability of downstream task performance in an LLM trained on diverse web-scale corpora. We find that task scores frequently fluctuate throughout training, both at the aggregate and example levels. To address this instability, we investigate two post-hoc checkpoint integration methods: checkpoint averaging and ensemble, motivated by the hypothesis that aggregating neighboring checkpoints can reduce performance volatility. We demonstrate both empirically and theoretically that these methods improve downstream performance stability without requiring any changes to the training procedure.