Learning New Tasks from a Few Examples with Soft-Label Prototypes
This addresses the problem of learning new NLP tasks with limited data, offering an incremental improvement for researchers and practitioners in few-shot learning.
The paper tackles few-shot learning in NLP by proposing soft-label prototypes to capture class distributions, achieving superior performance on most tasks with very few examples (4, 8, 16 per class) while being parameter-efficient.
Existing approaches to few-shot learning in NLP rely on large language models (LLMs) and/or fine-tuning of these to generalise on out-of-distribution data. In this work, we propose a novel few-shot learning approach based on soft-label prototypes (SLPs) designed to collectively capture the distribution of different classes across the input domain space. We focus on learning previously unseen NLP tasks from very few examples (4, 8, 16) per class and experimentally demonstrate that our approach achieves superior performance on the majority of tested tasks in this data-lean setting while being highly parameter efficient. We also show that our few-shot adaptation method can be integrated into more generalised learning settings, primarily meta-learning, to yield superior performance against strong baselines.