ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue
This addresses the challenge of maintaining coherence and accuracy in multi-turn dialogues for users and developers of LLMs, representing an incremental improvement over existing context management methods.
The paper tackles the problem of LLMs struggling with alignment, dependencies, and factual drift in multi-turn dialogues by proposing the Adaptive Context Refactoring (ACR) Framework, which dynamically reshapes interaction history to mitigate contextual inertia and state drift, resulting in significant performance improvements and reduced token consumption in experiments.
Large Language Models (LLMs) have shown remarkable performance in multi-turn dialogue. However, in multi-turn dialogue, models still struggle to stay aligned with what has been established earlier, follow dependencies across many turns, and avoid drifting into incorrect facts as the interaction grows longer. Existing approaches primarily focus on extending the context window, introducing external memory, or applying context compression, yet these methods still face limitations such as \textbf{contextual inertia} and \textbf{state drift}. To address these challenges, we propose the \textbf{A}daptive \textbf{C}ontext \textbf{R}efactoring \textbf{(ACR)} Framework, which dynamically monitors and reshapes the interaction history to mitigate contextual inertia and state drift actively. ACR is built on a library of context refactoring operators and a teacher-guided self-evolving training paradigm that learns when to intervene and how to refactor, thereby decoupling context management from the reasoning process. Extensive experiments on multi-turn dialogue demonstrate that our method significantly outperforms existing baselines while reducing token consumption.