AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
This addresses the need for reliable monitoring and improvement of agentic systems in production, particularly for organizations deploying LLMs, though it appears incremental as it builds on existing evaluation concepts with a novel framework.
The paper tackles the problem of evaluating and debugging multi-agent LLM workflows in production, where current methods fail to capture errors and emergent behaviors, by introducing AgentCompass, a framework that achieves state-of-the-art results on the TRAIL benchmark and uncovers critical issues missed by humans.
With the growing adoption of Large Language Models (LLMs) in automating complex, multi-agent workflows, organizations face mounting risks from errors, emergent behaviors, and systemic failures that current evaluation methods fail to capture. We present AgentCompass, the first evaluation framework designed specifically for post-deployment monitoring and debugging of agentic workflows. AgentCompass models the reasoning process of expert debuggers through a structured, multi-stage analytical pipeline: error identification and categorization, thematic clustering, quantitative scoring, and strategic summarization. The framework is further enhanced with a dual memory system-episodic and semantic-that enables continual learning across executions. Through collaborations with design partners, we demonstrate the framework's practical utility on real-world deployments, before establishing its efficacy against the publicly available TRAIL benchmark. AgentCompass achieves state-of-the-art results on key metrics, while uncovering critical issues missed in human annotations, underscoring its role as a robust, developer-centric tool for reliable monitoring and improvement of agentic systems in production.