From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation
This work addresses the challenge of optimizing many-shot learning for LLM users, offering a method to enhance performance beyond simple scaling, though it is incremental as it builds on existing many-shot paradigms.
The paper tackles the problem of improving many-shot in-context learning by analyzing its factors and proposing an algorithm that identifies influential examples and regenerates new ones, leading to significant performance gains on tasks like symbolic reasoning, numerical reasoning, and code generation across various LLMs.
Recent advances in long-context large language models (LLMs) have led to the emerging paradigm of many-shot in-context learning (ICL), where it is observed that scaling many more demonstrating examples beyond the conventional few-shot setup in the context can lead to performance benefits. However, despite its promise, it is unclear what aspects dominate the benefits and whether simply scaling to more examples is the most effective way of improving many-shot ICL. In this work, we first provide an analysis of the factors driving many-shot ICL, and we find that 1) many-shot performance can still be attributed to often a few disproportionately influential examples and 2) identifying such influential examples ("optimize") and using them as demonstrations to regenerate new examples ("generate") can lead to further improvements. Inspired by the findings, we propose BRIDGE, an algorithm that alternates between the optimize step with Bayesian optimization to discover the influential sets of examples and the generate step to reuse this set to expand the reasoning paths of the examples back to the many-shot regime automatically. On Gemini, Claude, and Mistral LLMs of different sizes, we show that BRIDGE to significant improvements across a diverse set of tasks, including symbolic reasoning, numerical reasoning, and code generation.