Bumblebee: Foundation Model for Particle Physics Discovery
This addresses the challenge of analyzing particle physics data for discovering new particles, though it appears incremental as it adapts BERT-like methods to this domain.
The paper tackles the problem of particle physics discovery by introducing Bumblebee, a foundation model inspired by BERT that embeds particle 4-vectors and removes positional encodings for sequence-order invariance. It improves dileptonic top quark reconstruction resolution by 10-20% and achieves AUROC scores of 0.877 for toponium discrimination and 0.625 for initial state classification.
Bumblebee is a foundation model for particle physics discovery, inspired by BERT. By removing positional encodings and embedding particle 4-vectors, Bumblebee captures both generator- and reconstruction-level information while ensuring sequence-order invariance. Pre-trained on a masked task, it improves dileptonic top quark reconstruction resolution by 10-20% and excels in downstream tasks, including toponium discrimination (AUROC 0.877) and initial state classification (AUROC 0.625). The flexibility of Bumblebee makes it suitable for a wide range of particle physics applications, especially the discovery of new particles.