Code Models are Zero-shot Precondition Reasoners
This addresses the challenge of action plausibility for agents in environments like dialog and text-based games, though it is incremental as it builds on existing pre-trained code models and policy learning methods.
The paper tackled the problem of enabling agents to understand plausible actions in sequential decision-making tasks by using code representations to extract action preconditions from demonstrations in a zero-shot manner, resulting in enhanced performance for few-shot policy learning on task-oriented dialog and embodied textworld benchmarks.
One of the fundamental skills required for an agent acting in an environment to complete tasks is the ability to understand what actions are plausible at any given point. This work explores a novel use of code representations to reason about action preconditions for sequential decision making tasks. Code representations offer the flexibility to model procedural activities and associated constraints as well as the ability to execute and verify constraint satisfaction. Leveraging code representations, we extract action preconditions from demonstration trajectories in a zero-shot manner using pre-trained code models. Given these extracted preconditions, we propose a precondition-aware action sampling strategy that ensures actions predicted by a policy are consistent with preconditions. We demonstrate that the proposed approach enhances the performance of few-shot policy learning approaches across task-oriented dialog and embodied textworld benchmarks.