When Less is More: The LLM Scaling Paradox in Context Compression
This identifies a breakdown in scaling laws for faithful context preservation in open-ended generation, which is significant for researchers and practitioners in natural language processing and AI, though it is incremental as it complements existing evaluations in context compression.
The paper tackles the problem that larger language models, when used as compressors in a compressor-decoder setup, can paradoxically reduce the faithfulness of reconstructed contexts despite lower training loss, due to factors like knowledge overwriting and semantic drift. The result shows this Size-Fidelity Paradox occurs across models from 0.6B to 90B parameters, with concrete examples such as factual alterations and structural changes.
Scaling up model parameters has long been a prevalent training paradigm driven by the assumption that larger models yield superior generation capabilities. However, under lossy context compression in a compressor-decoder setup, we observe a Size-Fidelity Paradox: increasing the compressor size can lessen the faithfulness of reconstructed contexts though training loss decreases. Through extensive experiments across models from 0.6B to 90B, we coin this paradox arising from two dominant factors: 1) knowledge overwriting: larger models increasingly replace source facts with their own prior beliefs, e.g., ``the white strawberry'' $\to$ ``the red strawberry''; and 2) semantic drift: larger models tend to paraphrase or restructure content instead of reproducing it verbatim, e.g., ``Alice hit Bob'' $\to$ ``Bob hit Alice''. By holding model size fixed, we reflect on the emergent properties of compressed context representations. We show that the culprit is not parameter count itself, but the excessive semantic capacity and amplified generative uncertainty that accompany scaling. Specifically, the increased rank of context embeddings facilitates prior knowledge intrusion, whereas higher entropy over token prediction distributions promotes rewriting. Our results complement existing evaluations over context compression paradigm, underpinning a breakdown in scaling laws for faithful preservation in open-ended generation.