CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correction
This addresses the problem of privacy-preserving personalization for mobile users by enabling efficient fine-tuning of LLMs on devices, though it is incremental as it builds on federated learning and ZOO methods.
The paper tackles the challenge of fine-tuning large language models on resource-constrained mobile devices by proposing CooperLLM, a cloud-edge-end cooperative federated fine-tuning framework that uses zeroth-order optimization on devices with cloud-guided gradient correction. It reduces on-device memory by up to 86.4%, accelerates convergence by 8.8 times, and improves accuracy by up to 10 percentage points over baselines.
Large Language Models (LLMs) perform well on many NLP tasks, but fine-tuning them on resource-constrained mobile devices is challenging due to high memory and computation costs, despite growing demands for privacy-preserving personalization. Federated Learning (FL) enables local-data training, yet existing methods either rely on memory-intensive backpropagation or use zeroth-order optimization (ZOO), which avoids backward passes but suffers from slow convergence and degraded accuracy. We propose CooperLLM, a cloud-assisted edge-end cooperative federated fine-tuning framework that combines ZOO on mobile devices with cloud-guided gradient rectification. Mobile clients perform lightweight ZOO updates on private data, while the cloud fine-tunes on auxiliary public data using backpropagation and injects guided perturbations to rectify local updates, improving convergence and accuracy without violating privacy. To address system bottlenecks, CooperLLM introduces pipeline scheduling and adaptive compression to overlap computation and communication and reduce memory usage. Experiments on multiple Transformer models and datasets show that CooperLLM reduces on-device memory by up to $86.4\%$, accelerates convergence by $8.8 \times$, and improves accuracy by up to 10 percentage points over state-of-the-art ZOO-based baselines.