Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
This work addresses a practical deployment bottleneck for reasoning LLMs by reducing memory usage and increasing throughput, offering an incremental improvement through a training-free compression method.
The paper tackles the problem of inefficient inference in reasoning-focused language models due to lengthy intermediate reasoning paths, proposing Reasoning Path Compression (RPC) to accelerate generation by compressing the KV cache based on semantic sparsity, resulting in up to 1.60× throughput improvement with a 1.2% accuracy drop on the AIME 2024 benchmark.
Recent reasoning-focused language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers. While this approach is effective in solving problems that require logical thinking, long reasoning paths significantly increase memory usage and reduce throughput of token generation, limiting the practical deployment of such models. We propose Reasoning Path Compression (RPC), a training-free method that accelerates inference by leveraging the semantic sparsity of reasoning paths. RPC periodically compresses the KV cache by retaining cache entries that receive high importance score, which are computed using a selector window composed of recently generated queries. Experiments show that RPC improves generation throughput of QwQ-32B by up to 1.60$\times$ compared to the inference with full KV cache, with an accuracy drop of 1.2\% on the AIME 2024 benchmark. Our findings demonstrate that semantic sparsity in reasoning traces can be effectively exploited for compression, offering a practical path toward efficient deployment of reasoning LLMs. Our code is available at https://github.com/jiwonsong-dev/ReasoningPathCompression.