Leveraging Web-Crawled Data for High-Quality Fine-Tuning
This addresses the challenge of high-quality fine-tuning for specific domains without relying on costly data sources, though it is incremental as it builds on existing web-crawled data methods.
The paper tackled the problem of fine-tuning large language models without expensive human-annotated or GPT-4 generated data by using web-crawled data, achieving a 9.4% average score improvement in Chinese math problems and outperforming larger open-source and closed-source models like GPT-3.5.
Most large language models are fine-tuned using either expensive human-annotated data or GPT-4 generated data which cannot guarantee performance in certain domains. We argue that although the web-crawled data often has formatting errors causing semantic inaccuracies, it can still serve as a valuable source for high-quality supervised fine-tuning in specific domains without relying on advanced models like GPT-4. To this end, we create a paired training dataset automatically by aligning web-crawled data with a smaller set of high-quality data. By training a language model on this dataset, we can convert web data with irregular formats into high-quality ones. Our experiments show that training with the model-transformed data yields better results, surpassing training with only high-quality data by an average score of 9.4% in Chinese math problems. Additionally, our 7B model outperforms several open-source models larger than 32B and surpasses well-known closed-source models such as GPT-3.5, highlighting the efficacy of our approach.