No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation
For LLM users relying on retrieval-augmented generation, this work provides a practical decoding method to ensure context use does not degrade already-correct outputs, addressing a key reliability issue.
The paper addresses neutral regression in context-conditioned LLMs, where models overwrite correct outputs even with non-informative contexts. The proposed NWCAD method prevents this regression while maintaining accuracy gains on helpful contexts, achieving do-no-harm reliability without sacrificing performance.
Large language models (LLMs) can answer questions and summarize documents when conditioned on external contexts (e.g., retrieved evidence), yet context use remains unreliable: models may overwrite an already-correct output (neutral regression) even when the context is non-informative. We formalize neutral regression as a do-no-harm requirement and quantify it by measuring accuracy drops on baseline-correct items under answer-consistent contexts. We propose No-Worse Context-Aware Decoding (NWCAD), a decode-time adapter built on a two-stream setup with a two-stage gate: it backs off to no-context decoding when the context is non-informative, and otherwise uses context-conditioned decoding with a CAD-style fallback under uncertainty. We evaluate NWCAD on benchmarks that separate do-no-harm reliability from context utilization (accuracy gains on genuinely helpful contexts). NWCAD prevents neutral regression on baseline-correct items while preserving strong context-driven accuracy on helpful contexts.