Toward a Science of Intent: Closure Gaps and Delegation Envelopes for Open-World AI Agents
This work provides a theoretical framework for making AI agents safe and reliable in open-world settings, which is a key problem for real-world deployment of autonomous systems.
The paper addresses the challenge of deploying capable AI models in open-world institutions by proposing 'intent compilation'—transforming partially specified human purpose into inspectable artifacts. It formalizes residual openness as a closure-gap vector and delegation envelopes, distinguishing misclosure from undersearch, and outlines benchmark metrics for testing when closure interventions outperform additional inference-time search.
Recent work has framed intelligence in verifiable tasks as reducing time-to-solution through learned structure and test-time search, while systems work has explored learned runtimes in which computation, memory and I/O migrate into model state. These perspectives do not explain why capable models remain difficult to deploy in open institutions. We propose intent compilation: the transformation of partially specified human purpose into inspectable artifacts that bind execution. The relevant deployment distinction is closed-world solver versus open-world agent. In closed worlds, a checker is largely given; in open worlds, verification is distributed across semantic, evidentiary, procedural and institutional dimensions. Weformalize this residual openness as a closure-gap vector, define delegation envelopes as pre-authorized regions of action space, distinguish misclosure from undersearch, and outline benchmark metrics for testing when closure interventions outperform additional inference-time search.