Zero-Label Prompt Selection
This addresses the challenge of cross-task generalization for users of large language models in low-resource settings, though it is incremental as it builds on existing prompt-based approaches.
The paper tackles the problem of selecting high-performing prompts for large language models without labeled data, proposing a Zero-Label Prompt Selection method that improves zero-label performance by a sizeable margin over prior methods.
Natural language prompts have been shown to facilitate cross-task generalization for large language models. However, with no or limited labeled examples, the cross-task performance is highly sensitive to the choice of prompts, while selecting a high-performing prompt is challenging given the scarcity of labels. To address the issue, we propose a Zero-Label Prompt Selection (ZPS) method that selects prompts without any labeled data or gradient update. Specifically, given the candidate human-written prompts for a task, ZPS labels a set of unlabeled data with a prompt ensemble and uses the pseudo-labels for prompt selection. Experiments show that ZPS improves over prior methods by a sizeable margin in zero-label performance. We also extend ZPS to a few-shot setting and show its advantages over strong baselines such as prompt tuning and model tuning.