AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Web
For researchers studying decentralized information access, this benchmark provides the first systematic evaluation of multi-agent coordination in web tasks, highlighting gaps and opportunities compared to centralized retrieval.
AgentWebBench benchmarks multi-agent coordination in the Agentic Web paradigm, where user agents interact with website-specific content agents. Across four tasks and seven LLMs, multi-agent coordination lags behind centralized retrieval but can outperform it on question answering with larger models, and the benchmark reveals traffic concentration, test-time scaling benefits, and planning needs.
Agentic Web is an emerging paradigm where autonomous agents help users use online information. As the paradigm develops, content providers are also deploying agents to manage their data and serve it through controlled interfaces. This shift moves information access from centralized retrieval to decentralized coordination. To study this setting, we introduce AgentWebBench, a benchmark that evaluates how well a user agent synthesizes answers by interacting with website-specific content agents. We evaluate four tasks that cover common web information needs, spanning ranked retrieval (web search, web recommendation) and open-ended synthesis (question answering, deep research). Across seven advanced LLMs and three coordination strategies, multi-agent coordination generally lags behind centralized retrieval as expected, because user agent cannot directly access the corpus, but the gap shrinks with model scale and can even outperform centralized retrieval on question answering. This benchmark also enables us to study properties of the emerging paradigm of the digital world. We find that decentralized access concentrates traffic toward a small set of websites, test time scaling improves both interaction reliability and task performance, and strong results require sufficient interactions guided by careful planning. Finally, our failure analysis suggests that user agents need better planning and answer synthesis, while content agents need more reliable retrieval and evidence quality. Code, data, and APIs are released on https://github.com/cxcscmu/AgentWebBench.