Alpha-Trimming: Locally Adaptive Tree Pruning for Random Forests
This addresses the problem of optimizing random forest performance for practitioners by introducing a locally adaptive pruning method, which is incremental as it builds on existing random forest frameworks.
The paper tackles the problem of improving random forest predictive performance by adaptively pruning individual regression trees, contrary to the conventional wisdom of fully growing them. The result shows that mean squared prediction error is often substantially lowered on 46 benchmark data sets, without substantial increases compared to standard random forests.
We demonstrate that adaptively controlling the size of individual regression trees in a random forest can improve predictive performance, contrary to the conventional wisdom that trees should be fully grown. A fast pruning algorithm, alpha-trimming, is proposed as an effective approach to pruning trees within a random forest, where more aggressive pruning is performed in regions with a low signal-to-noise ratio. The amount of overall pruning is controlled by adjusting the weight on an information criterion penalty as a tuning parameter, with the standard random forest being a special case of our alpha-trimmed random forest. A remarkable feature of alpha-trimming is that its tuning parameter can be adjusted without refitting the trees in the random forest once the trees have been fully grown once. In a benchmark suite of 46 example data sets, mean squared prediction error is often substantially lowered by using our pruning algorithm and is never substantially increased compared to a random forest with fully-grown trees at default parameter settings.