Greedy feature selection: Classifier-dependent feature selection via greedy methods
This addresses the need for more effective feature selection in classification tasks, particularly for domain-specific applications like solar physics, but it is incremental as it builds on existing greedy methods by incorporating classifier dependence.
The paper tackles the problem of classifier-independent feature selection by introducing greedy feature selection, which selects features based on the classifier used, and shows theoretical benefits in model capacity indicators like VC dimension and kernel alignment, with numerical tests applied to predicting solar geo-effective manifestations.
The purpose of this study is to introduce a new approach to feature ranking for classification tasks, called in what follows greedy feature selection. In statistical learning, feature selection is usually realized by means of methods that are independent of the classifier applied to perform the prediction using that reduced number of features. Instead, greedy feature selection identifies the most important feature at each step and according to the selected classifier. In the paper, the benefits of such scheme are investigated theoretically in terms of model capacity indicators, such as the Vapnik-Chervonenkis (VC) dimension or the kernel alignment, and tested numerically by considering its application to the problem of predicting geo-effective manifestations of the active Sun.