Estimating the Operating Characteristics of Ensemble Methods
This work addresses computational efficiency in ensemble methods for researchers and practitioners, but it is incremental as it builds on existing bootstrap and random forest techniques.
The authors tackled the problem of efficiently estimating the operating characteristics of ensemble methods using a bootstrap-based technique, demonstrating that alternative meta-parameter selections for random forests can improve predictive accuracy.
In this paper we present a technique for using the bootstrap to estimate the operating characteristics and their variability for certain types of ensemble methods. Bootstrapping a model can require a huge amount of work if the training data set is large. Fortunately in many cases the technique lets us determine the effect of infinite resampling without actually refitting a single model. We apply the technique to the study of meta-parameter selection for random forests. We demonstrate that alternatives to bootstrap aggregation and to considering \sqrt{d} features to split each node, where d is the number of features, can produce improvements in predictive accuracy.