Multi-Programming Language Sandbox for LLMs
This tool simplifies and automates workflows for researchers and developers working on LLM-based code-related tasks, reducing development costs, though it is incremental as it builds on existing sandbox and analysis techniques.
The authors tackled the problem of providing unified feedback for code generated by Large Language Models (LLMs) by introducing MPLSandbox, a multi-programming language sandbox that automatically identifies, compiles, and executes code in isolated environments, integrating both traditional and LLM-based analysis tools to improve code quality and correctness.
We introduce MPLSandbox, an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). It can automatically identify the programming language of the code, compiling and executing it within an isolated sub-sandbox to ensure safety and stability. In addition, MPLSandbox also integrates both traditional and LLM-based code analysis tools, providing a comprehensive analysis of generated code. MPLSandbox can be effortlessly integrated into the training and deployment of LLMs to improve the quality and correctness of their generated code. It also helps researchers streamline their workflows for various LLM-based code-related tasks, reducing the development cost. To validate the effectiveness of MPLSandbox, we integrate it into training and deployment approaches, and also employ it to optimize workflows for a wide range of real-world code-related tasks. Our goal is to enhance researcher productivity on LLM-based code-related tasks by simplifying and automating workflows through delegation to MPLSandbox.