Federated Data-Efficient Instruction Tuning for Large Language Models
This work addresses data efficiency and privacy concerns in federated learning for large language model tuning, offering a practical solution for scenarios with limited or private data, though it is incremental as it builds on existing federated and data-efficient methods.
The paper tackles the problem of excessive computational overhead and overfitting in federated instruction tuning of large language models by proposing FedHDS, which uses a representative subset of edge-side data, achieving an average 10.72% improvement in Rouge-L on unseen tasks while using less than 1.5% of data samples and improving training efficiency by up to tens of times.
Instruction tuning is a crucial step in improving the responsiveness of pretrained large language models (LLMs) to human instructions. Federated learning (FL) helps to exploit the use of vast private instruction data from clients, becoming popular for LLM tuning by improving data diversity. Existing federated tuning simply consumes all local data, causing excessive computational overhead and overfitting to local data, while centralized data-efficient solutions are not suitable for FL due to privacy concerns. This work presents FedHDS, a federated data-efficient instruction tuning approach, which tunes LLMs with a representative subset of edge-side data. It reduces the data redundancy at both intra- and inter-client levels without sharing raw data. Experiments with various LLMs, datasets and partitions show that FedHDS improves Rouge-L on unseen tasks by an average of 10.72% over the SOTA full-data federated instruction tuning methods, while using less than 1.5% of the data samples, improving training efficiency by up to tens of times.