MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning
This work addresses efficiency and performance issues in few-shot learning for NLP practitioners, though it is incremental as it builds on existing prompt-based methods.
The paper tackles the performance limitations in few-shot prompt-based learning due to limited templates and text, introducing MixPro, an augmentation method that improves model performance by an average of 5.08% across five datasets.
Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cloze problems by combining original input with a predetermined template. This approach demonstrates its effectiveness, especially in few-shot learning scenarios, where the model is trained on a scarce amount of data. Despite its successes, the limited templates and text in few-shot prompt-based learning scenarios leave significant room for performance improvement. Moreover, existing methods sometimes resort to model ensembles, which, while effective, could potentially hamper model efficiency due to increased computational demands. To address these issues, we introduce MixPro, an augmentation method designed to augment both the vanilla input text and the templates. We implement this through the token-level, the sentence-level, and the template-level Mixup strategies. The experimental results on five few-shot datasets show that MixPro outperforms other augmentation baselines, improving model performance by an average of 5.08% compared to before augmentation.