Nonparametric Feature Selection by Random Forests and Deep Neural Networks
This work addresses a bottleneck in machine learning for practitioners dealing with high-dimensional data, though it is incremental as it builds on existing methods like random forests and deep neural networks.
The authors tackled the problem of computational inefficiency in random forests on large datasets with many irrelevant features by proposing a nonparametric feature selection algorithm that combines random forests and deep neural networks, demonstrating its advantage in identifying useful features and improving computation efficiency over alternatives in synthetic and real-world tests.
Random forests are a widely used machine learning algorithm, but their computational efficiency is undermined when applied to large-scale datasets with numerous instances and useless features. Herein, we propose a nonparametric feature selection algorithm that incorporates random forests and deep neural networks, and its theoretical properties are also investigated under regularity conditions. Using different synthetic models and a real-world example, we demonstrate the advantage of the proposed algorithm over other alternatives in terms of identifying useful features, avoiding useless ones, and the computation efficiency. Although the algorithm is proposed using standard random forests, it can be widely adapted to other machine learning algorithms, as long as features can be sorted accordingly.