Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
This addresses a critical issue for AI researchers and practitioners using synthesized data to scale training, offering a practical solution to maintain model performance.
The paper tackles the problem of model collapse in large language models trained on synthesized data by proposing verification to select high-quality generated examples, showing that verifiers prevent performance drops and that a proxy measure correlates with effectiveness in tasks like matrix eigenvalue computation and news summarization.
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is used as a replacement for expensive human-annotation. This raises concerns about \emph{model collapse}, a drop in model performance when their training sets include generated data. Considering that it is easier for both humans and machines to tell between good and bad examples than to generate high-quality samples, we investigate the use of verification on synthesized data to prevent model collapse. We provide a theoretical characterization using Gaussian mixtures, linear classifiers, and linear verifiers to derive conditions with measurable proxies to assess whether the verifier can effectively select synthesized data that leads to optimal performance. We experiment with two practical tasks -- computing matrix eigenvalues with transformers and news summarization with LLMs -- which both exhibit model collapse when trained on generated data, and show that verifiers, even imperfect ones, can indeed be harnessed to prevent model collapse and that our proposed proxy measure strongly correlates with performance.