Modeling Uncertainty Trends for Timely Retrieval in Dynamic RAG
This addresses a central challenge in dynamic RAG for improving LLM adaptability, though it is incremental as it builds on existing uncertainty-based methods.
The paper tackles the problem of determining optimal retrieval timing in dynamic retrieval-augmented generation (RAG) to prevent delayed intervention after errors propagate, introducing the Entropy-Trend Constraint (ETC) method that consistently outperforms baselines on six QA benchmarks while reducing retrieval frequency.
Dynamic retrieval-augmented generation (RAG) allows large language models (LLMs) to fetch external knowledge on demand, offering greater adaptability than static RAG. A central challenge in this setting lies in determining the optimal timing for retrieval. Existing methods often trigger retrieval based on low token-level confidence, which may lead to delayed intervention after errors have already propagated. We introduce Entropy-Trend Constraint (ETC), a training-free method that determines optimal retrieval timing by modeling the dynamics of token-level uncertainty. Specifically, ETC utilizes first- and second-order differences of the entropy sequence to detect emerging uncertainty trends, enabling earlier and more precise retrieval. Experiments on six QA benchmarks with three LLM backbones demonstrate that ETC consistently outperforms strong baselines while reducing retrieval frequency. ETC is particularly effective in domain-specific scenarios, exhibiting robust generalization capabilities. Ablation studies and qualitative analyses further confirm that trend-aware uncertainty modeling yields more effective retrieval timing. The method is plug-and-play, model-agnostic, and readily integrable into existing decoding pipelines. Implementation code is included in the supplementary materials.