Evaluating Agentic Harness Systems for Autonomous Computational Pathology
For computational pathology researchers, ACP-Bench provides a standardized evaluation framework to audit agentic systems before claiming clinical autonomy, though the results show current systems are far from reliable.
The paper introduces ACP-Bench, a benchmark for evaluating agentic harness systems on 41 pathology workflow tasks. Across 369 trajectories from 9 models and 3 harness groups, end-to-end completion remained rare, with tool execution and workflow revision being less mature than task interpretation and reporting.
Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.