debug-gym: A Text-Based Environment for Interactive Debugging
This addresses the need for more effective LLM agents in coding and debugging, though it is incremental as it builds on existing interactive agent frameworks.
The authors tackled the problem of LLMs lacking interactive exploration capabilities for coding tasks by introducing debug-gym, a text-based environment with tools like a Python debugger, which enables LLM-based agents to gather relevant information interactively.
Large Language Models (LLMs) are increasingly relied upon for coding tasks, yet in most scenarios it is assumed that all relevant information can be either accessed in context or matches their training data. We posit that LLMs can benefit from the ability to interactively explore a codebase to gather the information relevant to their task. To achieve this, we present a textual environment, namely debug-gym, for developing LLM-based agents in an interactive coding setting. Our environment is lightweight and provides a preset of useful tools, such as a Python debugger (pdb), designed to facilitate an LLM-based agent's interactive debugging. Beyond coding and debugging tasks, this approach can be generalized to other tasks that would benefit from information-seeking behavior by an LLM agent.