Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training
This addresses a critical risk for accuracy-dependent domains like AI training and knowledge-intensive applications, offering practical guidance, though it is incremental in building on prior work on model collapse.
The paper tackles the problem of knowledge collapse in large language models under recursive synthetic training, where factual accuracy deteriorates while fluency persists, leading to 'confidently wrong' outputs; they demonstrate that collapse depends on instruction format and propose domain-specific synthetic training as a mitigation strategy, achieving substantial improvements in collapse resistance.
Large language models increasingly rely on synthetic data due to human-written content scarcity, yet recursive training on model-generated outputs leads to model collapse, a degenerative process threatening factual reliability. We define knowledge collapse as a distinct three-stage phenomenon where factual accuracy deteriorates while surface fluency persists, creating "confidently wrong" outputs that pose critical risks in accuracy-dependent domains. Through controlled experiments with recursive synthetic training, we demonstrate that collapse trajectory and timing depend critically on instruction format, distinguishing instruction-following collapse from traditional model collapse through its conditional, prompt-dependent nature. We propose domain-specific synthetic training as a targeted mitigation strategy that achieves substantial improvements in collapse resistance while maintaining computational efficiency. Our evaluation framework combines model-centric indicators with task-centric metrics to detect distinct degradation phases, enabling reproducible assessment of epistemic deterioration across different language models. These findings provide both theoretical insights into collapse dynamics and practical guidance for sustainable AI training in knowledge-intensive applications where accuracy is paramount.