Empowering Sentence Encoders with Prompting and Label Retrieval for Zero-shot Text Classification
This work addresses the challenge of zero-shot text classification for NLP practitioners by improving label representation through retrieval, though it is incremental as it builds on existing sentence encoder methods.
The paper tackled the problem of adapting sentence encoders for zero-shot text classification by introducing RaLP, a framework that uses prompting and label retrieval to assign labels based on embedding similarity, achieving competitive or stronger performance than larger baselines on various datasets.
With contrastive pre-training, sentence encoders are generally optimized to locate semantically similar samples closer to each other in their embedding spaces. In this work, we focus on the potential of their embedding spaces to be readily adapted to zero-shot text classification, as semantically distinct samples are already well-separated. Our framework, RaLP (Retrieval augmented Label Prompts for sentence encoder), encodes prompted label candidates with a sentence encoder, then assigns the label whose prompt embedding has the highest similarity with the input text embedding. In order to compensate for the potentially poorly descriptive labels in their original format, RaLP retrieves sentences that are semantically similar to the original label prompt from external corpora and use them as additional pseudo-label prompts. RaLP achieves competitive or stronger performance than much larger baselines on various closed-set classification and multiple-choice QA datasets under zero-shot settings. We show that the retrieval component plays a pivotal role in RaLP's success, and its results are robustly attained regardless of verbalizer variations.