MMD-based Variable Importance for Distributional Random Forest
This work addresses the need for more comprehensive variable importance measures in distributional machine learning, offering a practical tool for feature selection in multivariate conditional distribution estimation, though it is incremental as it builds on existing forest-based methods.
The authors tackled the problem of identifying which input variables influence the entire conditional distribution of a multivariate output, not just its mean, by introducing a variable importance algorithm for Distributional Random Forests based on MMD distance and drop-and-relearn. They demonstrated that this algorithm is consistent, outperforms competitors in empirical tests on real and simulated data, and efficiently selects variables through recursive feature elimination to build accurate distribution estimates.
Distributional Random Forest (DRF) is a flexible forest-based method to estimate the full conditional distribution of a multivariate output of interest given input variables. In this article, we introduce a variable importance algorithm for DRFs, based on the well-established drop and relearn principle and MMD distance. While traditional importance measures only detect variables with an influence on the output mean, our algorithm detects variables impacting the output distribution more generally. We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors. In particular, our algorithm is highly efficient to select variables through recursive feature elimination, and can therefore provide small sets of variables to build accurate estimates of conditional output distributions.