Beam Search for Feature Selection
This addresses feature selection efficiency for machine learning practitioners, but it is incremental as it builds on existing methods like forward selection.
The paper tackles feature selection for classification by proposing beam search as a generalization of forward selection, showing it outperforms forward selection on correlated features and achieves comparable performance with only ten features instead of hundreds.
In this paper, we present and prove some consistency results about the performance of classification models using a subset of features. In addition, we propose to use beam search to perform feature selection, which can be viewed as a generalization of forward selection. We apply beam search to both simulated and real-world data, by evaluating and comparing the performance of different classification models using different sets of features. The results demonstrate that beam search could outperform forward selection, especially when the features are correlated so that they have more discriminative power when considered jointly than individually. Moreover, in some cases classification models could obtain comparable performance using only ten features selected by beam search instead of hundreds of original features.