A model-free feature selection technique of feature screening and random forest based recursive feature elimination
This provides a computationally efficient, nonparametric feature selection method for domains like multiclass classification and regression, but it is incremental as it builds on existing techniques like random forest and feature screening.
The authors tackled the problem of feature selection in ultra-high dimensional data by proposing a two-phase, model-free method combining fused Kolmogorov filter with random forest-based recursive feature elimination, achieving selection and L2 consistency under weak conditions and demonstrating superior performance in simulations and real data.
In this paper, we propose a model-free feature selection method for ultra-high dimensional data with mass features. This is a two phases procedure that we propose to use the fused Kolmogorov filter with the random forest based RFE to remove model limitations and reduce the computational complexity. The method is fully nonparametric and can work with various types of datasets. It has several appealing characteristics, i.e., accuracy, model-free, and computational efficiency, and can be widely used in practical problems, such as multiclass classification, nonparametric regression, and Poisson regression, among others. We show that the proposed method is selection consistent and $L_2$ consistent under weak regularity conditions. We further demonstrate the superior performance of the proposed method over other existing methods by simulations and real data examples.