On the Stability of Prompt Ranking in Large Language Model Evaluation
For practitioners and researchers selecting prompts for LLMs, the paper highlights a critical reliability issue and offers a practical fix.
The paper shows that prompt rankings in LLM evaluation are often unstable under minor variations (e.g., random seeds, limited subsets), causing unreliable top-prompt selection. A stability-aware selection strategy using a lower confidence bound improves robustness.
Prompt-based interaction has become a dominant paradigm for using large language models (LLMs), where multiple candidate prompts are evaluated and the top-ranked one is selected for downstream use. This workflow implicitly assumes that prompt rankings are stable under minor variations in evaluation conditions. In this paper, we systematically study prompt ranking stability under common sources of variability, including random seeds and limited evaluation subsets. Across three open-weight LLMs and two benchmark tasks, we find that while overall rank correlations are often moderate to high, the identity of the top-performing prompt frequently changes, leading to unreliable selection decisions. To address this issue, we propose a simple stability-aware selection strategy based on a lower confidence bound, which accounts for both performance and variance. Our results show that this approach improves robustness in unstable settings while remaining competitive in more stable regimes. These findings highlight the importance of accounting for evaluation uncertainty in prompt selection and LLM benchmarking.