Rethinking the Instruction Quality: LIFT is What You Need
This addresses the challenge of improving instruction tuning for LLM developers, though it appears incremental as it builds on existing quality improvement methods.
The paper tackles the problem of instruction data quality in LLM tuning by proposing LIFT, a paradigm that broadens data distribution and reduces redundancy, resulting in LLMs achieving robust performance and surpassing some state-of-the-art results with limited high-quality data.
Instruction tuning, a specialized technique to enhance large language model (LLM) performance via instruction datasets, relies heavily on the quality of employed data. Existing quality improvement methods alter instruction data through dataset expansion or curation. However, the expansion method risks data redundancy, potentially compromising LLM performance, while the curation approach confines the LLM's potential to the original dataset. Our aim is to surpass the original data quality without encountering these shortcomings. To achieve this, we propose LIFT (LLM Instruction Fusion Transfer), a novel and versatile paradigm designed to elevate the instruction quality to new heights. LIFT strategically broadens data distribution to encompass more high-quality subspaces and eliminates redundancy, concentrating on high-quality segments across overall data subspaces. Experimental results demonstrate that, even with a limited quantity of high-quality instruction data selected by our paradigm, LLMs not only consistently uphold robust performance across various tasks but also surpass some state-of-the-art results, highlighting the significant improvement in instruction quality achieved by our paradigm.