Making AI Compliance Evidence Machine-Readable
For organizations building high-risk AI systems, this provides a concrete, open-source approach to operationalize AI governance compliance, though it is an incremental application of an existing standard (OSCAL) to a new domain.
The paper addresses the lack of machine-readable evidence formats for AI compliance under frameworks like the EU AI Act. It proposes using OSCAL as an interchange format, defines 16 property extensions, and presents a Compliance-as-Code architecture that generates assurance evidence as a byproduct of model training, tested on two high-risk systems.
AI Assurance -- producing the machine-readable evidence required to demonstrate compliance with AI governance frameworks -- has mature policy scaffolding but lacks the infrastructure to operationalize it. Organizations building high-risk AI systems under the EU AI Act face a gap: frameworks such as the EU AI Act, ISO/IEC 42001, and NIST AI RMF specify what to assure but provide no executable format for how. This paper proposes OSCAL -- the NIST standard adopted for FedRAMP cybersecurity compliance -- as a candidate interchange format for AI governance, complementing rather than replacing the emerging JTC21 standards stack. We define 16 property extensions covering lifecycle phases, enforcement semantics, risk traceability, and risk-acceptance justification, and present a three-layer Compliance-as-Code architecture (policy, evidence, enforcement) that generates assurance evidence as a byproduct of model training. The SDK produces native OSCAL Assessment Results validated against the NIST JSON schema. We test the approach on two Annex III high-risk systems: a credit scoring model and a medical imaging segmentation system. The architecture and reference implementation are open-source under Apache 2.0.