Curriculum Demonstration Selection for In-Context Learning
This addresses the problem of optimizing demonstration selection for in-context learning in LLMs, offering a novel approach that enhances performance, particularly on challenging tasks.
The paper tackles the challenge of selecting demonstrations to maximize large language models' in-context learning potential by proposing Curriculum Demonstration Selection (CDS), which partitions samples by complexity and selects demonstrations from easy to difficult, resulting in consistent performance improvements across nine LLMs on three benchmarks.
Large Language Models (LLMs) have shown strong in-context learning (ICL) abilities with a few demonstrations. However, one critical challenge is how to select demonstrations to elicit the full potential of LLMs. In this paper, we propose Curriculum Demonstration Selection (CDS), a novel demonstration selection method for ICL. Instead of merely using similarity, CDS additionally partitions samples by their complexity measurements. Following curriculum learning, CDS then selects demonstrations from easy to difficult. Thus the selected demonstrations cover a wide range of difficulty levels, enabling LLMs to learn from varied complexities within the training set. Experiments demonstrate that our CDS consistently outperforms baseline methods, achieving notable improvements across nine LLMs on three benchmarks. Moreover, CDS proves especially effective in enhancing LLM performance in solving challenging problems.