Asymptotic Theory and Phase Transitions for Variable Importance in Quantile Regression Forests
This work addresses a fundamental problem for researchers and practitioners using random forests in high-dimensional settings, highlighting a trade-off between predictive performance and inferential validity, and is incremental in providing theoretical insights into existing methods.
The paper tackles the challenge of statistical inference for variable importance in Quantile Regression Forests by developing an asymptotic theory that reveals a phase transition phenomenon, showing that standard inference breaks down in bias-dominated regimes with large subsample sizes, and it derives the asymptotic bias to discuss restoring valid inference via bias correction.
Quantile Regression Forests (QRF) are widely used for non-parametric conditional quantile estimation, yet statistical inference for variable importance measures remains challenging due to the non-smoothness of the loss function and the complex bias-variance trade-off. In this paper, we develop a asymptotic theory for variable importance defined as the difference in pinball loss risks. We first establish the asymptotic normality of the QRF estimator by handling the non-differentiable pinball loss via Knight's identity. Second, we uncover a "phase transition" phenomenon governed by the subsampling rate $β$ (where $s \asymp n^β$). We prove that in the bias-dominated regime ($β\ge 1/2$), which corresponds to large subsample sizes typically favored in practice to maximize predictive accuracy, standard inference breaks down as the estimator converges to a deterministic bias constant rather than a zero-mean normal distribution. Finally, we derive the explicit analytic form of this asymptotic bias and discuss the theoretical feasibility of restoring valid inference via analytic bias correction. Our results highlight a fundamental trade-off between predictive performance and inferential validity, providing a theoretical foundation for understanding the intrinsic limitations of random forest inference in high-dimensional settings.