PIPer: On-Device Environment Setup via Online Reinforcement Learning
This addresses the problem of reducing manual effort in environment configuration for developers and scaling execution-based benchmarks for software engineering researchers, representing an incremental improvement over existing methods.
The paper tackles the problem of automating software environment setup, a persistent challenge in software engineering, by tuning a specialized model that combines supervised fine-tuning and reinforcement learning with verifiable rewards, enabling a smaller model (Qwen3-8B) to perform on par with larger models like Qwen3-32B and GPT-4o on the EnvBench-Python benchmark.
Environment setup-the process of configuring the system to work with a specific software project-represents a persistent challenge in Software Engineering (SE). Automated environment setup methods could assist developers by providing fully configured environments for arbitrary repositories without manual effort. This also helps SE researchers to scale execution-based benchmarks. However, recent studies reveal that even state-of-the-art Large Language Models (LLMs) achieve limited success in automating this task. To address this limitation, we tune a specialized model for environment setup. We combine supervised fine-tuning for generating correct Bash scripts and Reinforcement Learning with Verifiable Rewards (RLVR) to adapt it to the task of environment setup. On EnvBench-Python, our method enables Qwen3-8B (a model runnable on consumer hardware) to perform on par with larger models-Qwen3-32B and GPT-4o. The training code and model checkpoints are available online: https://github.com/JetBrains-Research/PIPer.