Grafting: Making Random Forests Consistent
It addresses a foundational theoretical gap in machine learning for researchers and practitioners using Random Forests, though it is incremental as it builds on existing literature.
The paper tackles the problem of Random Forests lacking theoretical consistency guarantees by grafting consistent estimators onto shallow CART trees, showing that this approach achieves consistency and performs well empirically.
Despite their performance and widespread use, little is known about the theory of Random Forests. A major unanswered question is whether, or when, the Random Forest algorithm is consistent. The literature explores various variants of the classic Random Forest algorithm to address this question and known short-comings of the method. This paper is a contribution to this literature. Specifically, the suitability of grafting consistent estimators onto a shallow CART is explored. It is shown that this approach has a consistency guarantee and performs well in empirical settings.