Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction
This addresses the reliability issue in zero-shot classification for users of large language models by reducing sensitivity to prompt variations, though it is an incremental improvement over existing methods.
The paper tackles the problem of prompt brittleness in zero-shot text classification by proposing Placeholding Parallel Prediction (P3), which predicts token probabilities across multiple positions to simulate generation paths, resulting in improved accuracy and up to a 98% reduction in standard deviation across prompts.
Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prompt can cause significant discrepancies in model performance. We attribute this prompt brittleness largely to the narrow focus on nexttoken probabilities in existing methods. To address this, we propose Placeholding Parallel Prediction (P3), a novel approach that predicts token probabilities across multiple positions and simulates comprehensive sampling of generation paths in a single run of a language model. Experiments show improved accuracy and up to 98% reduction in the standard deviation across prompts, boosting robustness. Even without a prompt, P3 maintains comparable performance, reducing the need for prompt engineering.