Personalized Collaborative Fine-Tuning for On-Device Large Language Models
This work addresses data heterogeneity and scarcity for on-device large language model fine-tuning, representing an incremental improvement over existing collaborative learning methods.
The paper tackles the problem of fine-tuning large language models on devices with limited and diverse local data by introducing trust-weighted gradient aggregation schemes and using Low-Rank Adaptation to reduce communication overhead. The results show that their protocols outperform FedAvg and local fine-tuning, especially in scenarios with heterogeneous data distributions.
We explore on-device self-supervised collaborative fine-tuning of large language models with limited local data availability. Taking inspiration from the collaborative learning community, we introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based. To minimize communication overhead, we integrate Low-Rank Adaptation (LoRA) and only exchange LoRA weight updates. Our protocols, driven by prediction and performance metrics, surpass both FedAvg and local fine-tuning methods, which is particularly evident in realistic scenarios with more diverse local data distributions. The results underscore the effectiveness of our approach in addressing heterogeneity and scarcity within local datasets.