Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks
This work addresses a bottleneck in NLU tasks, particularly those with large label counts, by innovatively extending prompting to labels, though it is incremental in building on existing prompt-tuning methods.
The paper tackles the problem of underutilizing semantically rich label names in natural language understanding tasks by introducing a Mask Matching method that applies prompts to both inputs and labels, achieving state-of-the-art performances on several datasets across 8 tasks.
Textual label names (descriptions) are typically semantically rich in many natural language understanding (NLU) tasks. In this paper, we incorporate the prompting methodology, which is widely used to enrich model input, into the label side for the first time. Specifically, we propose a Mask Matching method, which equips an input with a prompt and its label with another, and then makes predictions by matching their mask representations. We evaluate our method extensively on 8 NLU tasks with 14 datasets. The experimental results show that Mask Matching significantly outperforms its counterparts of fine-tuning and conventional prompt-tuning, setting up state-of-the-art performances in several datasets. Mask Matching is particularly good at handling NLU tasks with large label counts and informative label names. As pioneering efforts that investigate the label-side prompt, we also discuss open issues for future study.