Cascade: Token-Sharded Private LLM Inference
This addresses privacy concerns for users of third-party LLM inference services by offering a scalable solution, though it trades off cryptographic guarantees for performance, making it incremental in the field of secure inference.
The paper tackles the problem of private inference for large language models (LLMs) in third-party services by proposing Cascade, a multi-party protocol that uses token sharding to maintain privacy with improved performance, achieving orders of magnitude faster speeds than existing secure schemes.
As LLMs continue to increase in parameter size, the computational resources required to run them are available to fewer parties. Therefore, third-party inference services -- where LLMs are hosted by third parties with significant computational resources -- are becoming increasingly popular. However, third party inference raises critical concerns about user data privacy. To mitigate these risks, privacy researchers have developed provably secure schemes for third-party inference, such as Secure Multi-Party Computation (SMPC). However, SMPC protocols have significant computational and communication overhead, and do not scale to large models. In this work, we propose a new multi-party inference protocol, Cascade, that avoids these punitive costs by leveraging sharding in the sequence dimension to maintain privacy, trading off cryptographic privacy guarantees for increased performance and scalability. We demonstrate that Cascade is resistant to a generalization of a recent attack that is highly effective against other statistical privacy schemes, and that it is further resistant to learning-based attacks. As Cascade is orders of magnitude faster than existing schemes, our findings offer practical solutions for secure deployment of modern state-of-the-art LLMs.