Your Dense Retriever is Secretly an Expeditious Reasoner
This work addresses efficiency and accuracy challenges in retrieval systems for applications like search engines, though it is incremental as it builds on existing dense retrieval and LLM methods.
The paper tackled the problem of dense retrievers struggling with reasoning-intensive queries by proposing Adaptive Query Reasoning (AdaQR), a hybrid framework that dynamically routes queries to either fast dense reasoning or deep LLM reasoning, resulting in a 28% reduction in reasoning cost and a 7% improvement in retrieval performance on the BRIGHT benchmark.
Dense retrievers enhance retrieval by encoding queries and documents into continuous vectors, but they often struggle with reasoning-intensive queries. Although Large Language Models (LLMs) can reformulate queries to capture complex reasoning, applying them universally incurs significant computational cost. In this work, we propose Adaptive Query Reasoning (AdaQR), a hybrid query rewriting framework. Within this framework, a Reasoner Router dynamically directs each query to either fast dense reasoning or deep LLM reasoning. The dense reasoning is achieved by the Dense Reasoner, which performs LLM-style reasoning directly in the embedding space, enabling a controllable trade-off between efficiency and accuracy. Experiments on large-scale retrieval benchmarks BRIGHT show that AdaQR reduces reasoning cost by 28% while preserving-or even improving-retrieval performance by 7%.