FedsLLM: Federated Split Learning for Large Language Models over Communication Networks
This work addresses communication efficiency for large language models in wireless networks, representing an incremental improvement by integrating existing techniques like LoRA and splitfed learning.
The paper tackled the challenge of deploying large language models in wireless networks by proposing FedsLLM, a framework combining LoRA with splitfed learning to reduce processing loads and optimize training delays, achieving a 47.63% average reduction in delays compared to unoptimized scenarios.
Addressing the challenges of deploying large language models in wireless communication networks, this paper combines low-rank adaptation technology (LoRA) with the splitfed learning framework to propose the federated split learning for large language models (FedsLLM) framework. The method introduced in this paper utilizes LoRA technology to reduce processing loads by dividing the network into client subnetworks and server subnetworks. It leverages a federated server to aggregate and update client models. As the training data are transmitted through a wireless network between clients and both main and federated servers, the training delay is determined by the learning accuracy and the allocation of communication bandwidth. This paper models the minimization of the training delay by integrating computation and communication optimization, simplifying the optimization problem into a convex problem to find the optimal solution. Additionally, it presents a lemma that describes the precise solutions to this problem. Simulation results demonstrate that the proposed optimization algorithm reduces delays by an average of 47.63% compared to unoptimized scenarios.