Dissecting the Impact of Model Misspecification in Data-driven Optimization
This work addresses the problem of guiding prescriptive machine learning usage for researchers and practitioners by clarifying when integrated optimization methods outperform traditional ones, though it is incremental in nature.
The paper analyzes the performance of data-driven optimization methods under model misspecification, showing that an integrated estimation-optimization approach provides a universal double benefit in regret when models are misspecified, while traditional methods are better when models are nearly well-specified.
Data-driven optimization aims to translate a machine learning model into decision-making by optimizing decisions on estimated costs. Such a pipeline can be conducted by fitting a distributional model which is then plugged into the target optimization problem. While this fitting can utilize traditional methods such as maximum likelihood, a more recent approach uses estimation-optimization integration that minimizes decision error instead of estimation error. Although intuitive, the statistical benefit of the latter approach is not well understood yet is important to guide the prescriptive usage of machine learning. In this paper, we dissect the performance comparisons between these approaches in terms of the amount of model misspecification. In particular, we show how the integrated approach offers a ``universal double benefit'' on the top two dominating terms of regret when the underlying model is misspecified, while the traditional approach can be advantageous when the model is nearly well-specified. Our comparison is powered by finite-sample tail regret bounds that are derived via new higher-order expansions of regrets and the leveraging of a recent Berry-Esseen theorem.