MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
This addresses data scarcity in multi-task learning for LLM practitioners, though it is incremental as it builds on existing LoRA methods.
The paper tackles the problem of data inefficiency in multi-task fine-tuning for large language models using LoRA, introducing MeTA-LoRA, a two-stage framework that matches or surpasses traditional full-data LoRA performance with significantly less task-specific data.
Low-Rank Adaptation (LoRA) has emerged as one of the most widely used parameter-efficient fine-tuning (PEFT) methods for adapting large language models (LLMs) to downstream tasks. While highly effective in single-task settings, it struggles to efficiently leverage inter-task knowledge in complex multi-task learning scenarios, often requiring substantial task-specific data to achieve optimal performance. To address this limitation, we introduce MeTA-LoRA, a two-stage optimization framework that significantly improves data efficiency in multi-task adaptation. In the first stage, task-specific LoRA adapters are learned using only a few samples from each involved dataset, enabling rapid adaptation without large-scale supervision. In the second stage, the shared LoRA adapter is updated by aggregating gradients from multiple tasks to promote knowledge transfer across tasks, further reducing data usage by leveraging common patterns. In both multi-task learning and multilingual learning scenarios, our method matches or surpasses the performance of traditional full-data LoRA fine-tuning approaches, while using significantly less task-specific data.