The Platonic Universe: Do Foundation Models See the Same Sky?
This work addresses the problem of whether foundation models can unify astronomical data representations, potentially enabling reuse of pre-trained general-purpose architectures, though it is incremental in nature.
The study tested the Platonic Representation Hypothesis in astronomy by measuring representational convergence across foundation models trained on diverse astronomical data, finding that alignment increases with model capacity, supporting a shared representation of galaxy astrophysics.
We test the Platonic Representation Hypothesis (PRH) in astronomy by measuring representational convergence across a range of foundation models trained on different data types. Using spectroscopic and imaging observations from JWST, HSC, Legacy Survey, and DESI, we compare representations from vision transformers, self-supervised models, and astronomy-specific architectures via mutual $k$-nearest neighbour analysis. We observe consistent scaling: representational alignment generally increases with model capacity across our tested architectures, supporting convergence toward a shared representation of galaxy astrophysics. Our results suggest that astronomical foundation models can use pre-trained general-purpose architectures, allowing us to capitalise on the broader machine learning community's already-spent computational investment.