From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning
This work addresses the challenge of adapting plans in dynamic physical environments for embodied agents, providing a method that significantly improves recovery from execution failures.
DynamicEnvPlan is a closed-loop framework for long-horizon embodied planning that learns from execution failures to recover from environment perturbations. It boosts success rate from 33.3% to 76.2% across 104 task-scene combinations, improving all seven evaluation metrics including safety and affordance compliance.
Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring agents to adapt their plans under non-stationary conditions. We study this challenge through long-horizon embodied planning under environment deviations and execution uncertainty. Existing embodied-task benchmarks can expose such failures, but these failures are usually treated as evaluation outcomes instead of learnable signals for training agents to recover. In this work, we introduce DynamicEnvPlan, a closed-loop framework for high-level planning in dynamic environments. It extends embodied task execution with humanoid agents, high-level primitive skills, structured semantic memory, and controllable perturbations. Our data synthesis design consists of planning, perturbation, and guarded correction modules that turn dynamic execution states into recovery-oriented traces. The resulting traces are used for staged supervised fine-tuning, enabling the planner to learn from both nominal execution and perturbed recovery trajectories. Using 104 task-scene combinations spanning i.i.d., compositional generalization, and out-of-distribution settings for fine-tuning and evaluation, DynamicEnvPlan boosts success rate from 33.3% for the base planner to 76.2%, while improving across all seven evaluation metrics critical to physical-world interaction, including safety and affordance compliance.