Sparse Semantic Dimension as a Generalization Certificate for LLMs
This addresses a foundational theoretical problem in understanding LLM generalization for the ML/AI research community, offering a novel framework with potential safety applications.
The paper tackles the puzzle of why large language models generalize well despite having more parameters than training tokens, proposing that generalization is controlled by the sparse, low-dimensional geometry of internal representations rather than parameter count. They introduce Sparse Semantic Dimension (SSD) as a complexity measure, empirically validating it on GPT-2 Small and Gemma-2B with non-vacuous certificates and discovering a counter-intuitive scaling law where larger models require fewer calibration samples.
Standard statistical learning theory predicts that Large Language Models (LLMs) should overfit because their parameter counts vastly exceed the number of training tokens. Yet, in practice, they generalize robustly. We propose that the effective capacity controlling generalization lies in the geometry of the model's internal representations: while the parameter space is high-dimensional, the activation states lie on a low-dimensional, sparse manifold. To formalize this, we introduce the Sparse Semantic Dimension (SSD), a complexity measure derived from the active feature vocabulary of a Sparse Autoencoder (SAE) trained on the model's layers. Treating the LLM and SAE as frozen oracles, we utilize this framework to attribute the model's generalization capabilities to the sparsity of the dictionary rather than the total parameter count. Empirically, we validate this framework on GPT-2 Small and Gemma-2B, demonstrating that our bound provides non-vacuous certificates at realistic sample sizes. Crucially, we uncover a counter-intuitive "feature sharpness" scaling law: despite being an order of magnitude larger, Gemma-2B requires significantly fewer calibration samples to identify its active manifold compared to GPT-2, suggesting that larger models learn more compressible, distinct semantic structures. Finally, we show that this framework functions as a reliable safety monitor: out-of-distribution inputs trigger a measurable "feature explosion" (a sharp spike in active features), effectively signaling epistemic uncertainty through learned feature violation. Code is available at: https://github.com/newcodevelop/sparse-semantic-dimension.