Approximating Language Model Training Data from Weights
This addresses the issue of closed training data for researchers and practitioners, enabling data recovery from open weights, though it is incremental as it builds on existing gradient-based techniques.
The paper tackles the problem of approximating language model training data from model weights, proposing a gradient-based method that selects high-matching data from a public corpus to recover useful data, improving performance from 65% to 80% on AG News classification and reducing perplexity from 3.3 to 2.3 on MSMARCO.
Modern language models often have open weights but closed training data. We formalize the problem of data approximation from model weights and propose several baselines and metrics. We develop a gradient-based approach that selects the highest-matching data from a large public text corpus and show its effectiveness at recovering useful data given only weights of the original and finetuned models. Even when none of the true training data is known, our method is able to locate a small subset of public Web documents can be used to train a model to close to the original model performance given models trained for both classification and supervised-finetuning. On the AG News classification task, our method improves performance from 65% (using randomly selected data) to 80%, approaching the expert benchmark of 88%. When applied to a model trained with SFT on MSMARCO web documents, our method reduces perplexity from 3.3 to 2.3, compared to an expert LLAMA model's perplexity of 2.0.