A Cautionary Tale of Decorrelating Theory Uncertainties
This work highlights a critical pitfall for researchers in high-energy physics or similar fields using decorrelation for non-statistical uncertainties, cautioning against overreliance without proper decomposition.
The paper examines the use of decorrelation techniques to reduce theory uncertainties in machine learning classifiers, finding that while these methods can appear to reduce uncertainty, the actual uncertainty often remains much larger, as demonstrated with fragmentation modeling and higher-order corrections examples.
A variety of techniques have been proposed to train machine learning classifiers that are independent of a given feature. While this can be an essential technique for enabling background estimation, it may also be useful for reducing uncertainties. We carefully examine theory uncertainties, which typically do not have a statistical origin. We will provide explicit examples of two-point (fragmentation modeling) and continuous (higher-order corrections) uncertainties where decorrelating significantly reduces the apparent uncertainty while the actual uncertainty is much larger. These results suggest that caution should be taken when using decorrelation for these types of uncertainties as long as we do not have a complete decomposition into statistically meaningful components.