ENTMOOT: A Framework for Optimization over Ensemble Tree Models
This addresses the challenge of applying tree models to optimization tasks in industrial applications, offering a novel integration method.
The paper tackles the problem of using tree models for decision-making and black-box optimization by introducing ENTMOOT, a framework that integrates trained tree models with a reliable uncertainty measure and solves optimization problems with guaranteed global optimality, proving it as a strong competitor to existing frameworks.
Gradient boosted trees and other regression tree models perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. Despite their advantages, they are generally unpopular for decision-making tasks and black-box optimization, which is due to their difficult-to optimize structure and the lack of a reliable uncertainty measure. ENTMOOT is our new framework for integrating (already trained) tree models into larger optimization problems. The contributions of ENTMOOT include: (i) explicitly introducing a reliable uncertainty measure that is compatible with tree models, (ii) solving the larger optimization problems that incorporate these uncertainty aware tree models, (iii) proving that the solutions are globally optimal, i.e. no better solution exists. In particular, we show how the ENTMOOT approach allows a simple integration of tree models into decision-making and black-box optimization, where it proves as a strong competitor to commonly-used frameworks.