StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
This work addresses the problem of assessing LLM agents' ability to improve over time for AI researchers and developers, though it is incremental as it builds on existing agent capabilities by focusing on benchmarking.
The paper tackles the lack of benchmarks for evaluating continuous improvement in large language model (LLM) agents by introducing StreamBench, a pioneering benchmark that simulates an online learning environment with feedback streams, and it provides baselines and analysis to identify effective streaming strategies.
Recent works have shown that large language model (LLM) agents are able to improve themselves from experience, which is an important ability for continuous enhancement post-deployment. However, existing benchmarks primarily evaluate their innate capabilities and do not assess their ability to improve over time. To address this gap, we introduce StreamBench, a pioneering benchmark designed to evaluate the continuous improvement of LLM agents over an input-feedback sequence. StreamBench simulates an online learning environment where LLMs receive a continuous flow of feedback stream and iteratively enhance their performance. In addition, we propose several simple yet effective baselines for improving LLMs on StreamBench, and provide a comprehensive analysis to identify critical components that contribute to successful streaming strategies. Our work serves as a stepping stone towards developing effective online learning strategies for LLMs, paving the way for more adaptive AI systems in streaming scenarios. Source code: https://github.com/stream-bench/stream-bench. Benchmark website: https://stream-bench.github.io.