RINO: Renormalization Group Invariance with No Labels
This addresses the challenge of reliance on mismodeled simulations for labeled data in high energy physics, offering a method to enhance model robustness, though it appears incremental as it builds on existing self-supervised and transformer-based techniques.
The paper tackles the problem of domain shift in supervised machine learning for high energy physics by proposing RINO, a self-supervised learning approach that pretrains models directly on collision data to learn embeddings invariant to renormalization group flow scales, resulting in improved generalization from simulation training data to real collision data for jet identification tasks.
A common challenge with supervised machine learning (ML) in high energy physics (HEP) is the reliance on simulations for labeled data, which can often mismodel the underlying collision or detector response. To help mitigate this problem of domain shift, we propose RINO (Renormalization Group Invariance with No Labels), a self-supervised learning approach that can instead pretrain models directly on collision data, learning embeddings invariant to renormalization group flow scales. In this work, we pretrain a transformer-based model on jets originating from quantum chromodynamic (QCD) interactions from the JetClass dataset, emulating real QCD-dominated experimental data, and then finetune on the JetNet dataset -- emulating simulations -- for the task of identifying jets originating from top quark decays. RINO demonstrates improved generalization from the JetNet training data to JetClass data compared to supervised training on JetNet from scratch, demonstrating the potential for RINO pretraining on real collision data followed by fine-tuning on small, high-quality MC datasets, to improve the robustness of ML models in HEP.