LMD3: Language Model Data Density Dependence
This provides a framework for understanding how language model predictions depend on specific subsets of training data, which is useful for researchers and practitioners analyzing model behavior.
The researchers developed a methodology to analyze language model performance at the individual example level using training data density estimation, finding that increasing training data support for specific test queries increases density and predicts performance improvements, and that density measurements explain a significant fraction of variance in model perplexity.
We develop a methodology for analyzing language model task performance at the individual example level based on training data density estimation. Experiments with paraphrasing as a controlled intervention on finetuning data demonstrate that increasing the support in the training distribution for specific test queries results in a measurable increase in density, which is also a significant predictor of the performance increase caused by the intervention. Experiments with pretraining data demonstrate that we can explain a significant fraction of the variance in model perplexity via density measurements. We conclude that our framework can provide statistical evidence of the dependence of a target model's predictions on subsets of its training data, and can more generally be used to characterize the support (or lack thereof) in the training data for a given test task.