An Evaluation of Representation Learning Methods in Particle Physics Foundation Models
This provides reproducible baselines and a reference point for future foundation model development in particle physics, enabling more transparent progress across the community.
The researchers systematically evaluated different representation learning methods for particle physics foundation models using a unified transformer-based framework, finding that targeted supervised architectural modifications achieved state-of-the-art performance on benchmark jet classification tasks.
We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.