Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning
This addresses the efficiency and quality constraints in instruction tuning datasets for LLM developers, though it appears incremental as a filtering method.
The paper tackles the problem of inefficient instruction fine-tuning for Large Language Models by introducing Low-Confidence Gold (LCG), a filtering framework that selects valuable instruction pairs; it shows that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance with substantial improvements on MT-bench and consistent gains across metrics.
The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework's efficacy while maintaining model performance establishes a promising direction for efficient instruction tuning.