DF2: Distribution-Free Decision-Focused Learning
This addresses challenges in predict-then-optimize problems for machine learning practitioners, offering a novel approach to improve decision-making under uncertainty, though it appears incremental as it builds on existing DFL frameworks.
The paper tackles bottlenecks in decision-focused learning (DFL) under probabilistic settings, such as model mismatch and gradient approximation errors, by introducing DF2, a distribution-free method that directly learns the expected optimization function, and demonstrates its effectiveness on synthetic and real-world problems.
Decision-focused learning (DFL), which differentiates through the KKT conditions, has recently emerged as a powerful approach for predict-then-optimize problems. However, under probabilistic settings, DFL faces three major bottlenecks: model mismatch error, sample average approximation error, and gradient approximation error. Model mismatch error stems from the misalignment between the model's parameterized predictive distribution and the true probability distribution. Sample average approximation error arises when using finite samples to approximate the expected optimization objective. Gradient approximation error occurs when the objectives are non-convex and KKT conditions cannot be directly applied. In this paper, we present DF2, the first distribution-free decision-focused learning method designed to mitigate these three bottlenecks. Rather than depending on a task-specific forecaster that requires precise model assumptions, our method directly learns the expected optimization function during training. To efficiently learn this function in a data-driven manner, we devise an attention-based model architecture inspired by the distribution-based parameterization of the expected objective. We evaluate DF2 on two synthetic problems and three real-world problems, demonstrating the effectiveness of DF2. Our code is available at: https://github.com/Lingkai-Kong/DF2.