Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents
This addresses memory management for LLM agents in extended reasoning tasks, offering incremental improvements in efficiency for users handling long sessions.
The paper tackles the problem of large language models losing accumulated state when context windows reach limits, by proposing Contextual Memory Virtualisation (CMV) to manage session history as a DAG, resulting in token count reductions averaging 20% and up to 86% through structurally lossless trimming.
As large language models engage in extended reasoning tasks, they accumulate significant state -- architectural mappings, trade-off decisions, codebase conventions -- within the context window. This understanding is lost when sessions reach context limits and undergo lossy compaction. We propose Contextual Memory Virtualisation (CMV), a system that treats accumulated LLM understanding as version-controlled state. Borrowing from operating system virtual memory, CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives that enable context reuse across independent parallel sessions. We introduce a three-pass structurally lossless trimming algorithm that preserves every user message and assistant response verbatim while reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead by stripping mechanical bloat such as raw tool outputs, base64 images, and metadata. A single-user case-study evaluation across 76 real-world coding sessions demonstrates that trimming remains economically viable under prompt caching, with the strongest gains in mixed tool-use sessions, which average 39% reduction and reach break-even within 10 turns. A reference implementation is available at https://github.com/CosmoNaught/claude-code-cmv.