$D^2LoRA$: Data-Driven LoRA Initialization for Low Resource Tasks
This work addresses the challenge of adapting LLMs to multiple tasks with scarce data, reducing training expenses, but it is incremental as it builds on existing LoRA methods.
The paper tackled the problem of tuning large language models in low-resource scenarios by proposing D^2LoRA, a data-driven LoRA initialization method, which achieved a 1% improvement on the GSM8K benchmark and a 2-point improvement in ROUGE score for title generation tasks.
Tuning large language models is essential for optimizing their performance across diverse applications, particularly in scenarios with limited data availability. Tuning large language models in scarce data scenarios is crucial, particularly given that the convergence speed of the LoRA method is lower than that of full fine-tuning. In this paper, we present an analysis of post-training methods including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Odds Ratio Preference Optimization (ORPO) within the context of task-specific learning using the LoRA method. Next we introduce $D^2LoRA$, a data-driven approach for initializing LoRA metrics that enhances training efficiency, especially in limited-data settings. Our experiments compare $D^2LoRA$ with vanilla LoRA in terms of performance and catastrophic forgetting under extremely data-constrained conditions. The results demonstrate that $D^2LoRA$ achieves a 1% improvement GSM8K benchmark and a 2-point improvement in ROUGE score in title generation tasks. $D^2LoRA$ facilitates the adaptation of LLMs to multiple tasks even when task-specific data is scarce, thereby reducing training expenses and offering data cost.