VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs
This work addresses privacy and resource limitations for users deploying LLMs, offering a benchmark and framework for split learning, but it is incremental as it builds on existing collaborative learning methods.
The paper tackles the challenge of adapting Large Language Models (LLMs) to private domains under data privacy and computational constraints by introducing VFLAIR-LLM, a split learning framework that enables privacy-preserving inference and fine-tuning, benchmarking 5 attacks and 9 defenses across 18 datasets to provide practical recommendations.
With the advancement of Large Language Models (LLMs), LLM applications have expanded into a growing number of fields. However, users with data privacy concerns face limitations in directly utilizing LLM APIs, while private deployments incur significant computational demands. This creates a substantial challenge in achieving secure LLM adaptation under constrained local resources. To address this issue, collaborative learning methods, such as Split Learning (SL), offer a resource-efficient and privacy-preserving solution for adapting LLMs to private domains. In this study, we introduce VFLAIR-LLM (available at https://github.com/FLAIR-THU/VFLAIR-LLM), an extensible and lightweight split learning framework for LLMs, enabling privacy-preserving LLM inference and fine-tuning in resource-constrained environments. Our library provides two LLM partition settings, supporting three task types and 18 datasets. In addition, we provide standard modules for implementing and evaluating attacks and defenses. We benchmark 5 attacks and 9 defenses under various Split Learning for LLM(SL-LLM) settings, offering concrete insights and recommendations on the choice of model partition configurations, defense strategies, and relevant hyperparameters for real-world applications.