A Novel Prompt-tuning Method: Incorporating Scenario-specific Concepts into a Verbalizer
This work addresses a bottleneck in prompt-tuning for NLP researchers, offering an incremental improvement over existing verbalizer methods.
The paper tackled the problem of limited coverage and high bias in verbalizer construction for prompt-tuning by incorporating scenario-specific concepts, achieving state-of-the-art results on five zero-shot text classification datasets.
The verbalizer, which serves to map label words to class labels, is an essential component of prompt-tuning. In this paper, we present a novel approach to constructing verbalizers. While existing methods for verbalizer construction mainly rely on augmenting and refining sets of synonyms or related words based on class names, this paradigm suffers from a narrow perspective and lack of abstraction, resulting in limited coverage and high bias in the label-word space. To address this issue, we propose a label-word construction process that incorporates scenario-specific concepts. Specifically, we extract rich concepts from task-specific scenarios as label-word candidates and then develop a novel cascade calibration module to refine the candidates into a set of label words for each class. We evaluate the effectiveness of our proposed approach through extensive experiments on {five} widely used datasets for zero-shot text classification. The results demonstrate that our method outperforms existing methods and achieves state-of-the-art results.