SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models
This work addresses computational waste in domain-specific LLM adaptation, offering a more efficient fine-tuning method for practitioners in fields like agriculture and medicine, though it is incremental as it builds on existing supervised fine-tuning techniques.
The paper tackles the inefficiency of fine-tuning large language models (LLMs) on entire datasets when they already know much of the content, proposing a self-learning framework that filters out known QA pairs to focus on unknown knowledge, which reduces training time while achieving comparable performance gains in agriculture and medicine domains.
When using supervised fine-tuning (SFT) to adapt large language models (LLMs) to specific domains, a significant challenge arises: should we use the entire SFT dataset for fine-tuning? Common practice often involves fine-tuning directly on the entire dataset due to limited information on the LLM's past training data. However, if the SFT dataset largely overlaps with the model's existing knowledge, the performance gains are minimal, leading to wasted computational resources. Identifying the unknown knowledge within the SFT dataset and using it to fine-tune the model could substantially improve the training efficiency. To address this challenge, we propose a self-learning framework for LLMs inspired by human learning pattern. This framework takes a fine-tuning (SFT) dataset in a specific domain as input. First, the LLMs answer the questions in the SFT dataset. The LLMs then objectively grade the responses and filter out the incorrectly answered QA pairs. Finally, we fine-tune the LLMs based on this filtered QA set. Experimental results in the fields of agriculture and medicine demonstrate that our method substantially reduces training time while achieving comparable improvements to those attained with full dataset fine-tuning. By concentrating on the unknown knowledge within the SFT dataset, our approach enhances the efficiency of fine-tuning LLMs.