Entailment as Robust Self-Learner
This work addresses the challenge of making natural language understanding models more efficient and trustworthy for researchers and practitioners, though it is incremental as it builds on existing entailment pretraining methods.
The authors tackled the problem of improving zero-shot adaptation and robustness of pretrained entailment models for natural language understanding tasks by proposing a prompting strategy and a pseudo-label editing algorithm for self-training, resulting in more stable and robust performance on classification tasks compared to large language models.
Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits weakly supervised fine-tuning. In this work, we design a prompting strategy that formulates a number of different NLU tasks as contextual entailment. This approach improves the zero-shot adaptation of pretrained entailment models. Secondly, we notice that self-training entailment-based models with unlabeled data can significantly improve the adaptation performance on downstream tasks. To achieve more stable improvement, we propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training. We also found that both pretrained entailment-based models and the self-trained models are robust against adversarial evaluation data. Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.