Do Prompts Solve NLP Tasks Using Natural Language?
This work addresses the problem of selecting effective prompts for NLP practitioners, but it is incremental as it compares existing prompt types without introducing new methods.
The paper empirically compares three types of prompts (human-designed, schema, and null) for NLP tasks under few-shot and fully-supervised settings, finding that schema prompts are generally most effective, with performance gaps diminishing as training data increases.
Thanks to the advanced improvement of large pre-trained language models, prompt-based fine-tuning is shown to be effective on a variety of downstream tasks. Though many prompting methods have been investigated, it remains unknown which type of prompts are the most effective among three types of prompts (i.e., human-designed prompts, schema prompts and null prompts). In this work, we empirically compare the three types of prompts under both few-shot and fully-supervised settings. Our experimental results show that schema prompts are the most effective in general. Besides, the performance gaps tend to diminish when the scale of training data grows large.