Governance by Design: Architecting Agentic AI for Organizational Learning and Scalable Autonomy
For organizations deploying agentic AI, this paper provides practical lessons on embedding governance into system architecture to balance autonomy with accountability.
The paper examines how a large IT services company implemented governance for an agentic AI system through architectural and working arrangements, distilling seven lessons for building governance into such systems during operationalization and scaling.
Agentic AI systems - systems that can pursue goals through multi-step planning and tool-mediated action with limited direct supervision - are moving from experimental prototypes to enterprise deployments. This transition introduces tensions in implementation, scaling, and governance: organizations seek scalable autonomy for knowledge and coordination work, yet must preserve accountability, safety, cost control, and responsibility as systems initiate actions, access enterprise data, and evolve through iterative updates. Building on an in-depth qualitative case of a large IT services company's 2025 development and staged rollout of an agentic system integrated with enterprise tools; we show that governance is implemented through concrete architectural and working arrangements that determine what the system is allowed to do, which tools and data it can use, how memory is handled, and how performance improvements are introduced over time. We then distill seven lessons that explain how to build effective governance into agentic AI during operationalization and scaling.