EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
For researchers evaluating autonomous agents that improve policies through feedback, this work provides a controlled benchmark and diagnostic tools to disentangle policy evolution from open-ended software engineering progress.
The paper introduces Autonomous Policy Evolution, a controlled evaluation setting for agents that iteratively improve policies via feedback, and presents EvoPolicyGym, a benchmark of 16 interactive RL environments. GPT-5.5 achieves the strongest aggregate rank and top-two performance on all environments, and trajectory-level diagnostics reveal that success depends on discovering task-appropriate mechanisms and refining policies under bounded feedback.
Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.