Large Language Models Know What Makes Exemplary Contexts
This addresses the challenge of improving few-shot learning efficiency for users of large language models, though it appears incremental as it builds on existing in-context learning methods.
The paper tackles the problem of selecting optimal in-context examples for large language models by proposing a unified framework where models self-select, rank, and optimize demonstrations using reinforcement learning, resulting in enhanced in-context learning performance as validated experimentally.
In-context learning (ICL) has proven to be a significant capability with the advancement of Large Language models (LLMs). By instructing LLMs using few-shot demonstrative examples, ICL enables them to perform a wide range of tasks without needing to update millions of parameters. This paper presents a unified framework for LLMs that allows them to self-select influential in-context examples to compose their contexts; self-rank candidates with different demonstration compositions; self-optimize the demonstration selection and ordering through reinforcement learning. Specifically, our method designs a parameter-efficient retrieval head that generates the optimized demonstration after training with rewards from LLM's own preference. Experimental results validate the proposed method's effectiveness in enhancing ICL performance. Additionally, our approach effectively identifies and selects the most representative examples for the current task, and includes more diversity in retrieval.