Multivariate Prediction Intervals for Random Forests
This addresses the need for better uncertainty estimates in engineering and physical sciences where design optimization involves multiple competing objectives or constraints, though it appears incremental as it builds on existing bootstrap methods.
The paper tackles the problem of generating accurate multivariate prediction intervals for bagged models, such as random forests, to improve iterative design tasks like sequential learning with multiple correlated outputs. The result shows that the proposed recalibrated bootstrap method leads to a marked decrease in the number of iterations required to find satisfactory candidates in simulated scenarios.
Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, the design task depends on multiple correlated model outputs as objectives and/or constraints. To better solve these problems, we propose a recalibrated bootstrap method to generate multivariate prediction intervals for bagged models and show that it is well-calibrated. We apply the recalibrated bootstrap to a simulated sequential learning problem with multiple objectives and show that it leads to a marked decrease in the number of iterations required to find a satisfactory candidate. This indicates that the recalibrated bootstrap could be a valuable tool for practitioners using machine learning to optimize systems with multiple competing targets.