PatchWorld: Gradient-Free Optimization of Executable World Models
This work addresses the problem of inducing executable code as world models for prediction and planning under partial observability, offering a novel approach for agents in text-agent environments.
This paper introduces PatchWorld, a gradient-free framework that converts offline trajectories into executable Python world models using counterexample-guided code repair. PatchWorld-Simple achieved the highest code-based planning score among evaluated methods across seven AgentGym environments, reaching 76.4% macro success in live one-step lookahead without using LLM calls in the world-model prediction module.
Text-agent environments are typically modeled as partially observable Markov decision processes (POMDPs), assuming that the simulator's latent state and transition dynamics are hidden from the agent. Yet little work has examined whether executable code can be induced to serve as a world model for prediction and planning under partial observability. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair. Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score among evaluated methods, reaching 76.4\% macro success in live one-step lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility. This exposes a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.