Spin: An Efficient Secure Computation Framework with GPU Acceleration
This work addresses the problem of slow and inefficient secure computation for machine learning practitioners, offering incremental optimizations for specific functions like attention in Transformers.
Spin is a GPU-accelerated secure computation framework that tackles efficiency challenges in multi-party computation for machine learning, achieving up to 2x faster deep neural network training and improved efficiency for Transformer inference with 18.9 million parameters.
Accuracy and efficiency remain challenges for multi-party computation (MPC) frameworks. Spin is a GPU-accelerated MPC framework that supports multiple computation parties and a dishonest majority adversarial setup. We propose optimized protocols for non-linear functions that are critical for machine learning, as well as several novel optimizations specific to attention that is the fundamental unit of Transformer models, allowing Spin to perform non-trivial CNNs training and Transformer inference without sacrificing security. At the backend level, Spin leverages GPU, CPU, and RDMA-enabled smart network cards for acceleration. Comprehensive evaluations demonstrate that Spin can be up to $2\times$ faster than the state-of-the-art for deep neural network training. For inference on a Transformer model with 18.9 million parameters, our attention-specific optimizations enable Spin to achieve better efficiency, less communication, and better accuracy.