Lifted Representation Hypothesis in Language Models
For researchers studying LLM generalization and memory, this work identifies specific failure modes in rule learning, but the findings are incremental as they confirm known limitations without offering solutions.
The paper investigates how LLMs store and update knowledge, proposing that they use shared latent structures (lifting) rather than isolated facts. Through controlled experiments, they find LLMs prematurely lift patterns and fail to refine (shatter) when data has nested rules and exceptions.
Large language models (LLMs) often answer queries by mapping individual observations to more general rule-like structures. However, it remains unclear how these structures are stored, selected, and revised. To study this process, we propose thelifted representation hypothesis: LLMs update memory through shared latent structures rather than isolated instance-level facts. This view frames lifting as an efficient use of symmetry across instances, and shattering as the refinement of coarse lifted structures into more specific subtypes. We evaluate LLMs' lifting and shattering through controlled exception-learning experiments across in-context learning, LoRA, and full fine-tuning. We find that LLMs are vulnerable to shattering failures when data are governed by nested rules and exceptions, while lifting often occurs prematurely. These results highlight the need to study the relation between data and rule structures in LLMs.