Scalable and Efficient Hypothesis Testing with Random Forests
This provides a scalable and efficient testing method for researchers using random forests in scientific applications, such as ecology, though it is incremental as it builds on existing permutation and subsampling frameworks.
The paper tackles the problem of performing formal hypothesis testing for feature significance in random forests, which has been computationally prohibitive, and proposes a permutation-style test that is asymptotically valid, maintains high power, and reduces computations by orders of magnitude.
Throughout the last decade, random forests have established themselves as among the most accurate and popular supervised learning methods. While their black-box nature has made their mathematical analysis difficult, recent work has established important statistical properties like consistency and asymptotic normality by considering subsampling in lieu of bootstrapping. Though such results open the door to traditional inference procedures, all formal methods suggested thus far place severe restrictions on the testing framework and their computational overhead precludes their practical scientific use. Here we propose a permutation-style testing approach to formally assess feature significance. We establish asymptotic validity of the test via exchangeability arguments and show that the test maintains high power with orders of magnitude fewer computations. As importantly, the procedure scales easily to big data settings where large training and testing sets may be employed without the need to construct additional models. Simulations and applications to ecological data where random forests have recently shown promise are provided.