Agent Behavior Mining: Generative AI Agent Governance in Business Processes
For organizations deploying generative AI agents in business processes, this work addresses the governance challenge of non-deterministic agent behavior by providing a method to render it observable.
The paper introduces Agent Behavior Mining to make generative AI agent decision-making observable and traceable in business processes, enabling detection of policy deviations and quantification of operational variability. An exploratory study with 18 practitioners found that behavioral transparency is seen as a prerequisite for trust.
As organizations increasingly deploy generative AI agents to automate business processes, they face a governance dilemma: although these agents can increase operational flexibility, their non-deterministic nature challenges the control and standardization that Business Process Management seeks to enforce. This paper addresses this \emph{invisible autonomy risk} by introducing \emph{Agent Behavior Mining}, a governance capability that enables the application of process mining techniques to render generative AI agent decision-making observable and traceable. We (1) improve the understanding of generative AI agent behavior through an event data model that translates granular agent activities -- including reasoning traces, tool usage, and token costs -- into standardized process logs; (2) instantiate the data model in a multi-agent order-to-cash implementation, demonstrating how process managers can leverage agent logs to detect policy deviations and quantify operational variability; and (3) evaluate the perceived practical utility of the approach in an exploratory study with 18 industry practitioners. The results indicate that practitioners view behavioral transparency as a prerequisite for trust and consider the ability to examine agent reasoning as an important governance requirement for the next generation of AI-driven business processes.