Order Matters: Rethinking Prompt Construction in In-Context Learning
This work addresses prompt design for researchers and practitioners using in-context learning, revealing that ordering is as critical as selection, which is incremental but impactful for optimizing performance.
The paper challenges the assumption that example selection matters more than ordering in in-context learning, finding through experiments on classification and generation tasks with models up to 27B parameters and GPT-5 that ordering variance is comparable to selection variance, and strong orderings can be identified using a development set to achieve near-oracle performance.
In-context learning (ICL) enables large language models to perform new tasks by conditioning on a sequence of examples. Most prior work reasonably and intuitively assumes that which examples are chosen has a far greater effect on performance than how those examples are ordered, leading to a focus on example selection. We revisit this assumption and conduct a systematic comparison between the effect of selection and ordering. Through controlled experiments on both classification and generation tasks, using multiple open-source model families (0.5B to 27B parameters) and GPT-5, we find that the variance in performance due to different example orderings is comparable to that from using entirely different example sets. Furthermore, we show that strong orderings can be identified using only a development set, achieving performance close to an oracle that selects the best ordering based on test labels. Our findings highlight the equal and intertwined importance of example selection and ordering in prompt design, calling for a reexamination of the assumptions held in ICL.