Feature Selection with Distance Correlation
This addresses the problem of efficient and interpretable feature selection for physicists working with well-motivated features, though it is incremental as it builds on existing distance correlation concepts.
The authors tackled feature selection in machine learning by developing a new method based on Distance Correlation (DisCo), and demonstrated that it matches the performance of deeper architectures using only ten features and two orders-of-magnitude fewer parameters on boosted top- and W-tagging tasks.
Choosing which properties of the data to use as input to multivariate decision algorithms -- a.k.a. feature selection -- is an important step in solving any problem with machine learning. While there is a clear trend towards training sophisticated deep networks on large numbers of relatively unprocessed inputs (so-called automated feature engineering), for many tasks in physics, sets of theoretically well-motivated and well-understood features already exist. Working with such features can bring many benefits, including greater interpretability, reduced training and run time, and enhanced stability and robustness. We develop a new feature selection method based on Distance Correlation (DisCo), and demonstrate its effectiveness on the tasks of boosted top- and $W$-tagging. Using our method to select features from a set of over 7,000 energy flow polynomials, we show that we can match the performance of much deeper architectures, by using only ten features and two orders-of-magnitude fewer model parameters.