Multiple testing for signal-agnostic searches of new physics with machine learning
This addresses the challenge of model selection bias in physics searches, offering a method to improve robustness in detecting unknown signals, though it is incremental as it builds on existing likelihood-ratio test frameworks.
The paper tackles the problem of bias in machine learning-based hypothesis tests for signal-agnostic searches of new physics by proposing multiple testing strategies, showing that combining tests with different hyperparameters achieves performance comparable to the best single test while providing a more uniform response to anomalies.
In this work, we address the question of how to enhance signal-agnostic searches by leveraging multiple testing strategies. Specifically, we consider hypothesis tests relying on machine learning, where model selection can introduce a bias towards specific families of new physics signals. We show that it is beneficial to combine different tests, characterised by distinct choices of hyperparameters, and that performances comparable to the best available test are generally achieved while providing a more uniform response to various types of anomalies. Focusing on the New Physics Learning Machine, a methodology to perform a signal-agnostic likelihood-ratio test, we explore a number of approaches to multiple testing, such as combining p-values and aggregating test statistics.