Self-training Large Language Models through Knowledge Detection
This addresses the need for scalable and cost-effective training of LLMs by reducing dependency on large labeled datasets, though it appears incremental as it builds on existing self-training and consistency methods.
The paper tackles the problem of reducing hallucination in large language models by introducing a self-training paradigm where the model autonomously curates labels and selectively trains on unknown data, resulting in significant improvements in generation accuracy and mitigation of catastrophic forgetting in out-of-distribution benchmarks.
Large language models (LLMs) often necessitate extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. This paper explores a self-training paradigm, where the LLM autonomously curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. Empirical evaluations demonstrate significant improvements in reducing hallucination in generation across multiple subjects. Furthermore, the selective training framework mitigates catastrophic forgetting in out-of-distribution benchmarks, addressing a critical limitation in training LLMs. Our findings suggest that such an approach can substantially reduce the dependency on large labeled datasets, paving the way for more scalable and cost-effective language model training.