What Makes Pre-trained Language Models Better Zero-shot Learners?
This addresses the challenge of practical zero-shot learning for NLP applications where no labeled data is available, though it is incremental as it builds on existing prompt learning methods.
The paper tackles the problem of selecting effective prompt templates in zero-shot text classification without labeled data by proposing Perplexity Selection, which uses language discrepancy to forecast template performance, resulting in improved prediction accuracy in realistic zero-shot scenarios.
Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori. This is not ideal because in a realworld zero-shot scenario of practical relevance, no labelled data is available. Thus, we propose a simple yet effective method for screening reasonable prompt templates in zero-shot text classification: Perplexity Selection (Perplection). We hypothesize that language discrepancy can be used to measure the efficacy of prompt templates, and thereby develop a substantiated perplexity-based scheme allowing for forecasting the performance of prompt templates in advance. Experiments show that our method leads to improved prediction performance in a realistic zero-shot setting, eliminating the need for any labelled examples.