FLARE

“The generate-then-retrieve approach, while effective, is inefficient for queries that definitely need retrieval, as it introduces an extra generation step.”

“these methods overlook the potential noise introduced when handling multiple entities, which can degrade output quality”

“However, they both rely on numerous iterative retrieval and pseudo-generations, leading to significant computational costs as well.”

“FLARE initiates retrieval when any token in a generated sentence has a probability below a certain threshold”

“While effective, these approaches often add supervision, special control tokens, auxiliary probers, or multi-stage loops that increase engineering complexity and latency.”

“Although this method more precisely identifies the LLM's information needs, its efficacy heavily depends on meticulously crafted few-shot prompts brown2020languagemodelsfewshotlearners and requires continuous retrieval and refinement, leading to substantial manual effort and increased inference costs.”

“Although relying on low confidence token for retrieve seems intuitive, we argue that this often results in delayed retrieval, failing to intervene at the optimal moment.”

“However, focusing on local tokens may neglect global reasoning needs.”

“However, current dynamic RAG methods fail to predict whether the LLM has the capability to answer a question prior to generation, thereby triggering retrieval in advance. Moreover, most methods often rely on static rules, leading to ineffective timing for retrieval triggers during the generation process.”

“However, the former approach has limitations as LLMs tend to be overconfident, generating high-confidence probability distributions even when lacking relevant knowledge.”