SEO: Stochastic Experience Optimization for Large Language Models
This addresses the challenge of experience optimization for LLMs in specific tasks, offering a method to enhance performance without retraining, though it appears incremental as it builds on prior work in automatic experience finding.
The paper tackles the problem of automatically finding useful experiences for large language models (LLMs) to improve task performance without model parameter updates, proposing Stochastic Experience Optimization (SEO) to iteratively optimize model-specific experiences in natural language, with experiments on three tasks and three LLMs showing consistently improved performance and generalization to out-of-distribution data.
Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvious, since it is unclear what experiences suit specific LLMs. Previous studies intended to automatically find useful experiences using LLMs, while it is difficult to ensure the effectiveness of the obtained experience. In this paper, we propose Stochastic Experience Optimization (SEO), an iterative approach that finds optimized model-specific experience without modifying model parameters through experience update in natural language. In SEO, we propose a stochastic validation method to ensure the update direction of experience, avoiding unavailing updates. Experimental results on three tasks for three LLMs demonstrate that experiences optimized by SEO can achieve consistently improved performance. Further analysis indicates that SEO-optimized experience can generalize to out-of-distribution data, boosting the performance of LLMs on similar tasks.