Generalising Random Forest Parameter Optimisation to Include Stability and Cost
This work addresses the need for stable and cost-effective random forest predictions in industrial scenarios, though it is incremental as it builds on existing optimization methods.
The authors tackled the problem of optimizing random forest parameters for commercial applications by proposing a multi-criteria framework that balances prediction error, stability, and cost, demonstrating on real-world datasets that it leads to markedly different parameter settings compared to error-only optimization.
Random forests are among the most popular classification and regression methods used in industrial applications. To be effective, the parameters of random forests must be carefully tuned. This is usually done by choosing values that minimize the prediction error on a held out dataset. We argue that error reduction is only one of several metrics that must be considered when optimizing random forest parameters for commercial applications. We propose a novel metric that captures the stability of random forests predictions, which we argue is key for scenarios that require successive predictions. We motivate the need for multi-criteria optimization by showing that in practical applications, simply choosing the parameters that lead to the lowest error can introduce unnecessary costs and produce predictions that are not stable across independent runs. To optimize this multi-criteria trade-off, we present a new framework that efficiently finds a principled balance between these three considerations using Bayesian optimisation. The pitfalls of optimising forest parameters purely for error reduction are demonstrated using two publicly available real world datasets. We show that our framework leads to parameter settings that are markedly different from the values discovered by error reduction metrics.