Confidential Machine Learning within Graphcore IPUs
This addresses the need for secure, high-performance confidential computing in AI workloads, particularly for users of Graphcore IPUs, though it is incremental as it builds on existing trusted execution concepts.
The paper tackles the problem of enabling confidential machine learning on AI accelerators by presenting IPU Trusted Extensions (ITX), which provide strong confidentiality and integrity guarantees with less than 5% performance overhead and up to 17x better performance compared to CPU-based systems.
We present IPU Trusted Extensions (ITX), a set of experimental hardware extensions that enable trusted execution environments in Graphcore's AI accelerators. ITX enables the execution of AI workloads with strong confidentiality and integrity guarantees at low performance overheads. ITX isolates workloads from untrusted hosts, and ensures their data and models remain encrypted at all times except within the IPU. ITX includes a hardware root-of-trust that provides attestation capabilities and orchestrates trusted execution, and on-chip programmable cryptographic engines for authenticated encryption of code and data at PCIe bandwidth. We also present software for ITX in the form of compiler and runtime extensions that support multi-party training without requiring a CPU-based TEE. Experimental support for ITX is included in Graphcore's GC200 IPU taped out at TSMC's 7nm technology node. Its evaluation on a development board using standard DNN training workloads suggests that ITX adds less than 5% performance overhead, and delivers up to 17x better performance compared to CPU-based confidential computing systems relying on AMD SEV-SNP.