Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments
This addresses verification challenges in AI governance frameworks for model providers and auditors who need trustworthy safety evaluations.
The paper tackles the problem of unverifiable and non-confidential AI safety benchmarks by proposing Attestable Audits, which use Trusted Execution Environments to enable verifiable interactions with compliant AI models while protecting sensitive data, demonstrating feasibility with a prototype on Llama-3.1 benchmarks.
Benchmarks are important measures to evaluate safety and compliance of AI models at scale. However, they typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets. We propose Attestable Audits, which run inside Trusted Execution Environments and enable users to verify interaction with a compliant AI model. Our work protects sensitive data even when model provider and auditor do not trust each other. This addresses verification challenges raised in recent AI governance frameworks. We build a prototype demonstrating feasibility on typical audit benchmarks against Llama-3.1.