Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translation
This provides a scalable and efficient solution for improving instruction-following in LLMs, reducing reliance on costly manual annotations, though it is incremental as it builds on existing self-training and back-translation methods.
The paper tackles the problem of creating high-quality instruction-tuning datasets for large language models without manual annotations by introducing Kun, a self-training approach using instruction back-translation and answer polishment, which generated over a million Chinese instructional data points and demonstrated robustness and scalability in experiments with a 6B-parameter model.
In this paper, we introduce Kun, a novel approach for creating high-quality instruction-tuning datasets for large language models (LLMs) without relying on manual annotations. Adapting a self-training algorithm based on instruction back-translation and answer polishment, Kun leverages unlabelled data from diverse sources such as Wudao, Wanjuan, and SkyPile to generate a substantial dataset of over a million Chinese instructional data points. This approach significantly deviates from traditional methods by using a self-curation process to refine and select the most effective instruction-output pairs. Our experiments with the 6B-parameter Yi model across various benchmarks demonstrate Kun's robustness and scalability. Our method's core contributions lie in its algorithmic advancement, which enhances data retention and clarity, and its innovative data generation approach that substantially reduces the reliance on costly and time-consuming manual annotations. This methodology presents a scalable and efficient solution for improving the instruction-following capabilities of LLMs, with significant implications for their application across diverse fields. The code and dataset can be found at https://github.com/Zheng0428/COIG-Kun