Generative Adaptation of Dynamics to Environmental Shifts via Weight-space Diffusion
For machine learning practitioners dealing with dynamics prediction under distribution shifts, DynaDiff offers a computationally efficient alternative to fine-tuning, though the improvement is incremental over existing methods.
DynaDiff introduces a generative meta-learning framework that uses weight-space diffusion to adapt dynamics predictors to environmental shifts without fine-tuning, achieving a 10.78% average prediction accuracy improvement over baselines.
Data-driven dynamics prediction often fails under environmental shifts, while traditional fine-tuning remains computationally prohibitive for hardware-constrained or data-scarce applications. We propose DynaDiff, a generative meta-learning framework that transitions the paradigm from gradient-based tuning or modulation to direct weight-space generation. Specifically, we first abstract expert weights as novel weight graphs, utilizing multi-head attention to explicitly capture topological coupling within weights. Subsequently, we design a functional loss to ensure that the generated models achieve consistency with expert models in physical behavior. Finally, we develop a dynamics-informed prompter that extracts cross-domain physical and spectral features from observation sequences to condition the diffusion model. Experiments demonstrate that DynaDiff boosts average prediction accuracy by 10.78% over competitive baselines. Furthermore, by pre-constructing a model zoo of expert predictors, we amortize the fine-tuning overhead into a one-time offline cost, significantly boosting deployment efficiency in new environments.