Context Compression Is Not One Thing: Readable Symbolic Re-expression vs. Coherent Summary at Matched Budget
For practitioners of multi-hop QA with small language models, Telegraph English offers a simple compression method that preserves reasoning evidence better than coherent summaries or random subsampling at the same token budget.
The paper introduces Telegraph English, a readable symbolic format for compressing retrieved passages in multi-hop QA, achieving 13-20 F1 point gains over three matched-budget baselines on MuSiQue, TwoWiki, and HotpotQA.
We study context compression for multi-hop question answering with small language models. We propose Telegraph English, a readable symbolic format that rewrites retrieved passages into structured entity-relation statements, preserving reasoning evidence at lower token cost. In controlled experiments on MuSiQue, TwoWiki, and HotpotQA, Telegraph English outperforms three matched-budget compression baselines (character-level deletion, truncation, and random sub-sampling) on every dataset, with gains of 13 to 20 F1 percentage point. It also outperforms a coherent prose summary produced by the same encoder on the hardest dataset. A pre-registered depth-interaction hypothesis is null: the advantage does not grow with reasoning depth within datasets. We interpret these results as evidence that readable symbolic re-expression preserves entity content more densely than either natural language or coherent summarization at matched token budget.