LLM-in-Sandbox Elicits General Agentic Intelligence
This addresses the challenge of eliciting agentic capabilities in LLMs for real-world deployment across diverse domains, though it appears incremental as it builds on existing sandbox and reinforcement learning methods.
The paper tackles the problem of enabling large language models (LLMs) to exhibit general intelligence in non-code domains by allowing them to explore within a code sandbox, resulting in robust generalization across areas like mathematics, physics, and long-context understanding without additional training or with post-training using non-agentic data.
We introduce LLM-in-Sandbox, enabling LLMs to explore within a code sandbox (i.e., a virtual computer), to elicit general intelligence in non-code domains. We first demonstrate that strong LLMs, without additional training, exhibit generalization capabilities to leverage the code sandbox for non-code tasks. For example, LLMs spontaneously access external resources to acquire new knowledge, leverage the file system to handle long contexts, and execute scripts to satisfy formatting requirements. We further show that these agentic capabilities can be enhanced through LLM-in-Sandbox Reinforcement Learning (LLM-in-Sandbox-RL), which uses only non-agentic data to train models for sandbox exploration. Experiments demonstrate that LLM-in-Sandbox, in both training-free and post-trained settings, achieves robust generalization spanning mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following. Finally, we analyze LLM-in-Sandbox's efficiency from computational and system perspectives, and open-source it as a Python package to facilitate real-world deployment.