Manuel Costa

Manuel Costa, Boris Köpf, Aashish Kolluri et al.

As AI agents become increasingly autonomous and capable, ensuring their security against vulnerabilities such as prompt injection becomes critical. This paper explores the use of information-flow control (IFC) to provide security guarantees for AI agents. We present a formal model to reason about the security and expressiveness of agent planners. Using this model, we characterize the class of properties enforceable by dynamic taint-tracking and construct a taxonomy of tasks to evaluate security and utility trade-offs of planner designs. Informed by this exploration, we present Fides, a planner that tracks confidentiality and integrity labels, deterministically enforces security policies, and introduces novel primitives for selectively hiding information. Its evaluation in AgentDojo demonstrates that this approach enables us to complete a broad range of tasks with security guarantees. A tutorial to walk readers through the the concepts introduced in the paper can be found at https://github.com/microsoft/fides

Aashish Kolluri, Rishi Sharma, Manuel Costa et al.

Indirect prompt injection attacks threaten AI agents that execute consequential actions, motivating deterministic system-level defenses. Such defenses can provably block unsafe actions by enforcing confidentiality and integrity policies, but currently appear costly: they reduce task completion rates and increase token usage compared to probabilistic defenses. We argue that existing evaluations miss a key benefit of system-level defenses: reduced reliance on human oversight. We introduce autonomy metrics to quantify this benefit: the fraction of consequential actions an agent can execute without human-in-the-loop (HITL) approval while preserving security. To increase autonomy, we design a security-aware agent that (i) introduces richer HITL interactions, and (ii) explicitly plans for both task progress and policy compliance. We implement this agent design atop an existing information-flow control defense against prompt injection and evaluate it on the AgentDojo and WASP benchmarks. Experiments show that this approach yields higher autonomy without sacrificing utility.

Paul England, Henrique S. Malvar, Eric Horvitz et al.

Advances in graphics and machine learning have led to the general availability of easy-to-use tools for modifying and synthesizing media. The proliferation of these tools threatens to cast doubt on the veracity of all media. One approach to thwarting the flow of fake media is to detect modified or synthesized media through machine learning methods. While detection may help in the short term, we believe that it is destined to fail as the quality of fake media generation continues to improve. Soon, neither humans nor algorithms will be able to reliably distinguish fake versus real content. Thus, pipelines for assuring the source and integrity of media will be required---and increasingly relied upon. We propose AMP, a system that ensures the authentication of media via certifying provenance. AMP creates one or more publisher-signed manifests for a media instance uploaded by a content provider. These manifests are stored in a database allowing fast lookup from applications such as browsers. For reference, the manifests are also registered and signed by a permissioned ledger, implemented using the Confidential Consortium Framework (CCF). CCF employs both software and hardware techniques to ensure the integrity and transparency of all registered manifests. AMP, through its use of CCF, enables a consortium of media providers to govern the service while making all its operations auditable. The authenticity of the media can be communicated to the user via visual elements in the browser, indicating that an AMP manifest has been successfully located and verified.

Manuel Costa, Lawrence Esswood, Olga Ohrimenko et al.

Modern processors, e.g., Intel SGX, allow applications to isolate secret code and data in encrypted memory regions called enclaves. While encryption effectively hides the contents of memory, the sequence of address references issued by the secret code leaks information. This is a serious problem because these leaks can easily break the confidentiality guarantees of enclaves. In this paper, we explore Oblivious RAM (ORAM) designs that prevent these information leaks under the constraints of modern SGX processors. Most ORAMs are a poor fit for these processors because they have high constant overhead factors or require large private memories, which are not available in these processors. We address these limitations with a new hierarchical ORAM construction, the Pyramid ORAM, that is optimized towards online bandwidth cost and small blocks. It uses a new hashing scheme that circumvents the complexity of previous hierarchical schemes. We present an efficient x64-optimized implementation of Pyramid ORAM that uses only the processor's registers as private memory. We compare Pyramid ORAM with Circuit ORAM, a state-of-the-art tree-based ORAM scheme that also uses constant private memory. Pyramid ORAM has better online asymptotical complexity than Circuit ORAM. Our implementation of Pyramid ORAM and Circuit ORAM validates this: as all hierarchical schemes, Pyramid ORAM has high variance of access latencies; although latency can be high for some accesses, for typical configurations Pyramid ORAM provides access latencies that are 8X better than Circuit ORAM for 99% of accesses. Although the best known hierarchical ORAM has better asymptotical complexity, Pyramid ORAM has significantly lower constant overhead factors, making it the preferred choice in practice.