UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking
This work addresses a fundamental limitation in current LLM-based information-seeking agents, which are blind to unindexed information, impacting anyone relying on these agents for comprehensive information retrieval.
This paper identifies and explores the problem of Unindexed Information Seeking (UIS), where critical information is not discoverable by standard search engines. They introduce UIS-QA, the first benchmark for this problem, on which state-of-the-art agents drop from 70.90% to 24.55%. They propose UIS-Digger, a multi-agent framework that achieves a baseline of 27.27% on UIS-QA, outperforming systems using more sophisticated LLMs.
Recent advancements in LLM-based information-seeking agents have achieved record-breaking performance on established benchmarks. However, these agents remain heavily reliant on search-engine-indexed knowledge, leaving a critical blind spot: Unindexed Information Seeking (UIS). This paper identifies and explores the UIS problem, where vital information is not captured by search engine crawlers, such as overlooked content, dynamic webpages, and embedded files. Despite its significance, UIS remains an underexplored challenge. To address this gap, we introduce UIS-QA, the first dedicated UIS benchmark, comprising 110 expert-annotated QA pairs. Notably, even state-of-the-art agents experience a drastic performance drop on UIS-QA (e.g., from 70.90 on GAIA and 46.70 on BrowseComp-zh to 24.55 on UIS-QA), underscoring the severity of the problem. To mitigate this, we propose UIS-Digger, a novel multi-agent framework that incorporates dual-mode browsing and enables simultaneous webpage searching and file parsing. With a relatively small $\sim$30B-parameter backbone LLM optimized using SFT and RFT training strategies, UIS-Digger sets a strong baseline at 27.27\%, outperforming systems integrating sophisticated LLMs such as O3 and GPT-4.1. This demonstrates the importance of proactive interaction with unindexed sources for effective and comprehensive information-seeking. Our work not only uncovers a fundamental limitation in current agent evaluation paradigms but also provides the first toolkit for advancing UIS research, defining a new and promising direction for robust information-seeking systems.