SRSA: A Cost-Efficient Strategy-Router Search Agent for Real-world Human-Machine Interactions
This addresses the need for cost-efficient and effective search agents in authentic chatbot scenarios, though it appears incremental by building on existing agent frameworks.
The paper tackles the problem of LLM-based search agents struggling with contextual and personalized queries in real-world human-machine interactions by proposing SRSA, which routes queries to appropriate search strategies to achieve high-quality results at low cost, demonstrating improved performance on metrics like informativeness and completeness without fine-tuning.
Recently, as Large Language Models (LLMs) have shown impressive emerging capabilities and gained widespread popularity, research on LLM-based search agents has proliferated. In real-world situations, users often input contextual and highly personalized queries to chatbots, challenging LLMs to capture context and generate appropriate answers. However, much of the prior research has not focused specifically on authentic human-machine dialogue scenarios. It also ignores the important balance between response quality and computational cost by forcing all queries to follow the same agent process. To address these gaps, we propose a Strategy-Router Search Agent (SRSA), routing different queries to appropriate search strategies and enabling fine-grained serial searches to obtain high-quality results at a relatively low cost. To evaluate our work, we introduce a new dataset, Contextual Query Enhancement Dataset (CQED), comprising contextual queries to simulate authentic and daily interactions between humans and chatbots. Using LLM-based automatic evaluation metrics, we assessed SRSA's performance in terms of informativeness, completeness, novelty, and actionability. To conclude, SRSA provides an approach that resolves the issue of simple serial searches leading to degenerate answers for lengthy and contextual queries, effectively and efficiently parses complex user queries, and generates more comprehensive and informative responses without fine-tuning an LLM.