ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix
This addresses the challenge of efficiently utilizing noisy data for instruction alignment in LLMs, offering a novel approach to data enhancement.
The paper tackles the problem of low-quality data in supervised fine-tuning of large language models by introducing ENTP, a framework that purifies and reconstructs low-quality samples through neural-symbolic methods, resulting in datasets that outperform 13 baselines and even fine-tuning on the full original dataset of approximately 300K examples.
Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.