CoRect: Context-Aware Logit Contrast for Hidden State Rectification to Resolve Knowledge Conflicts
This addresses unfaithful outputs in RAG systems for users relying on accurate information retrieval, but it is incremental as it builds on existing layer-wise analysis and rectification methods.
The paper tackles the problem of knowledge conflicts in Retrieval-Augmented Generation (RAG), where internal model knowledge overrides retrieved evidence, by proposing CoRect to identify and rectify biased layers, resulting in improved faithfulness and reduced hallucinations across QA and summarization benchmarks.
Retrieval-Augmented Generation (RAG) often struggles with knowledge conflicts, where model-internal parametric knowledge overrides retrieved evidence, leading to unfaithful outputs. Existing approaches are often limited, relying either on superficial decoding adjustments or weight editing that necessitates ground-truth targets. Through layer-wise analysis, we attribute this failure to a parametric suppression phenomenon: specifically, in deep layers, certain FFN layers overwrite context-sensitive representations with memorized priors. To address this, we propose CoRect (Context-Aware Logit Contrast for Hidden State Rectification). By contrasting logits from contextualized and non-contextualized forward passes, CoRect identifies layers that exhibit high parametric bias without requiring ground-truth labels. It then rectifies the hidden states to preserve evidence-grounded information. Across question answering (QA) and summarization benchmarks, CoRect consistently improves faithfulness and reduces hallucinations compared to strong baselines.