Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making
This addresses the challenge of synthesizing reliable, feasible trajectories from fixed datasets in offline decision-making, with potential applications in robotics and control, though it appears incremental as it builds on existing diffusion and model-based methods.
The paper tackles the problem of generating dynamically feasible behaviors in offline decision-making by proposing MPDiffuser, a model-based diffusion framework that interleaves planning and dynamics updates during sampling, resulting in consistent gains over existing approaches on benchmarks like D4RL and DSRL and successful deployment on a real quadrupedal robot.
Offline decision-making requires synthesizing reliable behaviors from fixed datasets without further interaction, yet existing generative approaches often yield trajectories that are dynamically infeasible. We propose Model Predictive Diffuser (MPDiffuser), a compositional model-based diffusion framework consisting of: (i) a planner that generates diverse, task-aligned trajectories; (ii) a dynamics model that enforces consistency with the underlying system dynamics; and (iii) a ranker module that selects behaviors aligned with the task objectives. MPDiffuser employs an alternating diffusion sampling scheme, where planner and dynamics updates are interleaved to progressively refine trajectories for both task alignment and feasibility during the sampling process. We also provide a theoretical rationale for this procedure, showing how it balances fidelity to data priors with dynamics consistency. Empirically, the compositional design improves sample efficiency, as it leverages even low-quality data for dynamics learning and adapts seamlessly to novel dynamics. We evaluate MPDiffuser on both unconstrained (D4RL) and constrained (DSRL) offline decision-making benchmarks, demonstrating consistent gains over existing approaches. Furthermore, we present a preliminary study extending MPDiffuser to vision-based control tasks, showing its potential to scale to high-dimensional sensory inputs. Finally, we deploy our method on a real quadrupedal robot, showcasing its practicality for real-world control.