Evidence-Bound Gateway-Path Provenance for Third-Party LLM Inference
For clients using third-party LLM gateways, this work provides a mechanism to independently verify gateway behavior, addressing trust issues in the LLM inference supply chain.
The paper addresses the problem of clients being unable to verify the integrity of decisions made by third-party LLM gateways, such as routing, fallback, and billing. It presents an evidence-bound gateway architecture using an attested runtime that enforces policy and signs cryptographic evidence, with a Rust prototype on AWS Nitro Enclaves showing modest overhead and fail-closed detection of tampering.
Third-party LLM gateways have become a critical infrastructure layer between applications and external LLM providers. Conventional gateways do more than forward traffic: they decide which provider and model are called, whether fallback occurred, which stream is delivered, and what usage record should be billed. Because these decisions and records are authored inside the operator-controlled service, clients cannot independently distinguish honest mediation from route substitution, hidden fallback, stream manipulation, or forged provenance. We present an evidence-bound LLM gateway architecture that separates the operator control plane from an attested execution plane. Within the gateway, a measured Attested Gateway Runtime (AGR) is the only component allowed to decrypt requests, enforce path policy, construct upstream calls, and sign evidence. Clients verify signed release metadata and fresh attestation before encrypting requests to keys bound to the AGR measurement. AGR enforces request-scoped routing, fallback, and endpoint constraints, invokes admitted providers, returns encrypted response streams, and signs evidence binding the policy, selected route, endpoint identity, stream commitments, and completion metadata to the attested runtime. An initial Rust prototype on AWS Nitro Enclaves shows modest mechanism overhead and fail-closed detection of policy, routing, endpoint, and stream-evidence tampering outside the attested runtime.