Friend-training: Learning from Models of Different but Related Tasks
This work addresses the challenge of leveraging related tasks in NLP for better pseudo-label selection, offering a novel framework that could benefit multi-task learning scenarios, though it appears incremental as it builds on existing self-training methods.
The paper tackles the problem of improving model performance on related natural language processing tasks by proposing friend-training, a cross-task self-training framework where models for different tasks help each other through iterative pseudo-labeling and retraining. The result shows that models trained with this framework achieve the best performance on conversational semantic role labeling and dialogue rewriting tasks compared to strong baselines.
Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differences in input features, model architectures, and training processes. However, many tasks in natural language processing are about different but related aspects of language, and models trained for one task can be great teachers for other related tasks. In this work, we propose friend-training, a cross-task self-training framework, where models trained to do different tasks are used in an iterative training, pseudo-labeling, and retraining process to help each other for better selection of pseudo-labels. With two dialogue understanding tasks, conversational semantic role labeling and dialogue rewriting, chosen for a case study, we show that the models trained with the friend-training framework achieve the best performance compared to strong baselines.