Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation
This work addresses the issue of weak prompt-demonstration associations in few-shot learning for NLP, offering an incremental improvement over existing methods.
The paper tackles the problem of demonstration learning in few-shot settings by introducing Imitation-Demo, which enhances prompt-demonstration dependencies through explicit imitation of human review behavior, achieving state-of-the-art performance on 11 out of 14 classification corpora.
Demonstration learning aims to guide the prompt prediction via providing answered demonstrations in the few shot settings. Despite achieving promising results, existing work only concatenates the answered examples as demonstrations to the prompt template (including the raw context) without any additional operation, neglecting the prompt-demonstration dependencies. Besides, prior research found that randomly replacing the labels of demonstrations marginally hurts performance, illustrating that the model could not properly learn the knowledge brought by the demonstrations. Inspired by the human learning process, in this paper, we introduce Imitation DEMOnstration Learning (Imitation-Demo) to strengthen demonstration learning via explicitly imitating human review behaviour, which includes: (1) contrastive learning mechanism to concentrate on the similar demonstrations. (2) demonstration-label re-prediction method to consolidate known knowledge. Experiment results show that our proposed method achieves state-of-the-art performance on 11 out of 14 classification corpora. Further studies also prove that Imitation-Demo strengthen the association between prompt and demonstrations, which could provide the basis for exploring how demonstration learning works.