PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics
For reinforcement learning and robotics, this work addresses the lack of physical structure in world models, improving both performance and internal simulator fidelity.
PH-Dreamer introduces a physics-driven world model using a Port-Hamiltonian framework to enforce physical priors in latent dynamics, achieving superior returns, tighter reward alignment, and reductions in phase space volume (4.18-8.41%), energy consumption (up to 7.80%), and jerk (up to 9.38%) on visual control benchmarks.
World models built on recurrent state space architectures enable efficient latent imagination, yet remain physically unstructured, producing dynamics that violate conservation and dissipative principles. We introduce a unified Port-Hamiltonian framework that remedies this through three synergistic mechanisms. First, we embed implicit physical priors into recurrent transitions by modeling projected latent evolution as action controlled energy routing governed by flow and dissipation, biasing the projected PH phase space toward a more compact and physically structured representation. Second, we develop a kinematics aware energy world model that estimates the Hamiltonian and power balance from proprioceptive observations, providing an explicit physical signal for thermodynamic reasoning. Third, leveraging these energy gradients, we establish an energy guided Actor-Critic that uses Lagrangian multipliers to regularize policy optimization toward lower energy and smoother control. Across visual control benchmarks, this paradigm not only attains superior asymptotic returns but also elevates internal simulator fidelity by establishing a tighter, lower variance alignment between imagined and real rewards, all while reducing latent phase space volume by 4.18-8.41%, energy consumption by up to 7.80%, and mean squared jerk by up to 9.38%.