Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded
This work addresses the challenge of optimizing discrete models in decision-making for industries like chemical engineering, though it appears incremental as it builds on existing optimization techniques for tree-based models.
The paper tackles the problem of integrating pre-trained gradient-boosted trees into mixed-integer nonlinear nonconvex optimization problems, such as chemical process catalyst selection, by developing heuristic and exact branch-and-bound algorithms. It tests these methods on concrete mixture design and chemical catalysis instances, showing computational feasibility for industrially-relevant applications.
Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pre-trained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models, or they may wish to optimize a discrete model that particularly well-represents a data set. We develop several heuristic methods to find feasible solutions, and an exact, branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on concrete mixture design instance and a chemical catalysis industrial instance.