Transfer Learning Beyond the Standard Model
This addresses the high computational cost of simulations for cosmologists, though it is incremental as it builds on existing transfer learning methods in a specific domain.
The paper tackles the problem of reducing simulation costs for cosmological inference by applying transfer learning from the standard ΛCDM model to various beyond-ΛCDM scenarios, showing it enables inference with significantly fewer simulations but also risks negative transfer due to physical degeneracies.
Machine learning enables powerful cosmological inference but typically requires many high-fidelity simulations covering many cosmological models. Transfer learning offers a way to reduce the simulation cost by reusing knowledge across models. We show that pre-training on the standard model of cosmology, $Λ$CDM, and fine-tuning on various beyond-$Λ$CDM scenarios -- including massive neutrinos, modified gravity, and primordial non-Gaussianities -- can enable inference with significantly fewer beyond-$Λ$CDM simulations. However, we also show that negative transfer can occur when strong physical degeneracies exist between $Λ$CDM and beyond-$Λ$CDM parameters. We consider various transfer architectures, finding that including bottleneck structures provides the best performance. Our findings illustrate the opportunities and pitfalls of foundation-model approaches in physics: pre-training can accelerate inference, but may also hinder learning new physics.