Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions
This work addresses the problem of unreliable zero-shot classification for NLP practitioners by introducing a more robust and context-aware method, though it is incremental as it builds on existing prompting paradigms.
The paper tackles the issues of miscalibration and brittleness in language model prompting for NLP tasks by proposing Gen-Z, a generative prompting framework for zero-shot text classification that conditions on natural language label descriptions. The result shows that this approach consistently outperforms zero-shot and few-shot baselines on standard benchmarks, improving robustness to prompt variations.
Language model (LM) prompting--a popular paradigm for solving NLP tasks--has been shown to be susceptible to miscalibration and brittleness to slight prompt variations, caused by its discriminative prompting approach, i.e., predicting the label given the input. To address these issues, we propose Gen-Z--a generative prompting framework for zero-shot text classification. GEN-Z is generative, as it measures the LM likelihood of input text, conditioned on natural language descriptions of labels. The framework is multivariate, as label descriptions allow us to seamlessly integrate additional contextual information about the labels to improve task performance. On various standard classification benchmarks, with six open-source LM families, we show that zero-shot classification with simple contextualization of the data source of the evaluation set consistently outperforms both zero-shot and few-shot baselines while improving robustness to prompt variations. Further, our approach enables personalizing classification in a zero-shot manner by incorporating author, subject, or reader information in the label descriptions.