Execution-Verified Reinforcement Learning for Optimization Modeling
This addresses the challenge of scalable decision intelligence for optimization practitioners by providing a more efficient and generalizable approach to modeling, though it is incremental in building on existing reinforcement learning and verification ideas.
The paper tackles the problem of automating optimization modeling with LLMs by proposing EVOM, a framework that uses execution-verified reinforcement learning to generate solver-specific code from natural-language problems, eliminating the need for process-level supervision and enabling cross-solver generalization. Experiments show EVOM matches or outperforms process-supervised methods and supports zero-shot solver transfer.
Automating optimization modeling with LLMs is a promising path toward scalable decision intelligence, but existing approaches either rely on agentic pipelines built on closed-source LLMs with high inference latency, or fine-tune smaller LLMs using costly process supervision that often overfits to a single solver API. Inspired by reinforcement learning with verifiable rewards, we propose Execution-Verified Optimization Modeling (EVOM), an execution-verified learning framework that treats a mathematical programming solver as a deterministic, interactive verifier. Given a natural-language problem and a target solver, EVOM generates solver-specific code, executes it in a sandboxed harness, and converts execution outcomes into scalar rewards, optimized with GRPO and DAPO in a closed-loop generate-execute-feedback-update process. This outcome-only formulation removes the need for process-level supervision, and enables cross-solver generalization by switching the verification environment rather than reconstructing solver-specific datasets. Experiments on NL4OPT, MAMO, IndustryOR, and OptiBench across Gurobi, OR-Tools, and COPT show that EVOM matches or outperforms process-supervised SFT, supports zero-shot solver transfer, and achieves effective low-cost solver adaptation by continuing training under the target solver backend.