AIMay 27

Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor

arXiv:2605.2871393.4
AI Analysis

For LLM inference acceleration, this work provides a new compression paradigm that leverages the intrinsic reasoning capabilities of thinking models, eliminating the need for complex compression modules or specialized training.

The paper reveals that thinking models can naturally compress long contexts by generating thinking traces, and proposes a reward-driven optimization framework (TaC-C) to achieve compact, controllable compression. At 4x and 8x compression ratios, TaC-C outperforms the strongest competitor by 17.4% and 23.4% in average F1, and by 15.7% and 21.7% in average EM on long-context QA benchmarks.

Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown promise, they typically rely on complex compression modules or compression-specific training, leaving the intrinsic capabilities of LLMs underexplored. In contrast, this work reveals that a thinking model itself can naturally compress long contexts by organizing task-relevant information. We thus derive Thinking as Compression (TaC), a new compression paradigm that treats thinking itself as compressed context. Without relying on specific dedicated compressor, TaC directly prompts the thinking model to generate thinking traces as the shortened context, already outperforming most representative compression methods. Further, given that raw thinking output may struggle with budget control and shortcut behaviors, we introduce Thinking as Compression Constrained (TaC-C), leveraging a simple reward-driven optimization framework to elicit intrinsic thinking as compact and controllable compressed context. Experiments across four long-context QA benchmarks demonstrate that TaC-C consistently outperforms existing baselines. At 4x and 8x compression ratios, it surpasses the strongest competitor by 17.4% and 23.4% in average F1, and by 15.7% and 21.7% in average Exact Match Score (EM), respectively.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes