Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation
This addresses the challenge of generating text from structured data with few examples, offering an incremental improvement over existing methods like task-adaptive pre-training.
The paper tackles the problem of limited labeled data in data-to-text generation by proposing Curriculum-Based Self-Training (CBST), which uses pseudo-labeled data in a difficulty-based order to improve few-shot learning, achieving state-of-the-art performance in this setting.
Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks. Existing works mostly utilize abundant unlabeled structured data to conduct unsupervised pre-training for task adaption, which fail to model the complex relationship between source structured data and target texts. Thus, we introduce self-training as a better few-shot learner than task-adaptive pre-training, which explicitly captures this relationship via pseudo-labeled data generated by the pre-trained model. To alleviate the side-effect of low-quality pseudo-labeled data during self-training, we propose a novel method called Curriculum-Based Self-Training (CBST) to effectively leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Experimental results show that our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.