Asymptotic Unbiasedness of the Permutation Importance Measure in Random Forest Models
This work provides theoretical guarantees for a widely used variable selection method, which is incremental as it builds on existing permutation importance measures in Random Forest models.
The paper tackles the problem of variable selection in sparse regression models, particularly in high-dimensional settings where the link function is not directly detectable, by proving the asymptotic unbiasedness of the permutation importance measure in Random Forest models under specific assumptions, with simulation studies verifying these findings.
Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative variables is challenging. Under these circumstances, the Random Forest method is a helpful tool to predict new outcomes while delivering measures for variable selection. One common approach is the usage of the permutation importance. Due to its intuitive idea and flexible usage, it is important to explore circumstances, for which the permutation importance based on Random Forest correctly indicates informative covariates. Regarding the latter, we deliver theoretical guarantees for the validity of the permutation importance measure under specific assumptions and prove its (asymptotic) unbiasedness. An extensive simulation study verifies our findings.