3D-Learning: Diffusion-Augmented Distributionally Robust Decision-Focused Learning
This addresses generalization challenges in computing and networked systems, offering a robust solution for decision-making tasks, though it appears incremental by building on distributionally robust optimization with diffusion models.
The paper tackles the problem of out-of-distribution (OOD) generalization in predict-then-optimize pipelines, such as cloud LLM serving, by proposing a diffusion-augmented distributionally robust learning framework that improves decision performance under worst-case distributions, outperforming existing methods in empirical tests on an LLM resource provisioning task.
Predict-then-Optimize (PTO) pipelines are widely employed in computing and networked systems, where Machine Learning (ML) models are used to predict critical contextual information for downstream decision-making tasks such as cloud LLM serving, data center demand response, and edge workload scheduling. However, these ML predictors are often vulnerable to out-of-distribution (OOD) samples at test time, leading to significant decision performance degradation due to large prediction errors. To address the generalization challenges under OOD conditions, we present the framework of Distributionally Robust Decision-Focused Learning (DR-DFL), which trains ML models to optimize decision performance under the worst-case distribution. Instead of relying on classical Distributionally Robust Optimization (DRO) techniques, we propose Diffusion-Augmented Distributionally Robust Decision-Focused Learning (3D-Learning), which searches for the worst-case distribution within the parameterized space of a diffusion model. By leveraging the powerful distribution modeling capabilities of diffusion models, 3D-Learning identifies worst-case distributions that remain consistent with real data, achieving a favorable balance between average and worst-case scenarios. Empirical results on an LLM resource provisioning task demonstrate that 3D-Learning outperforms existing DRO and Data Augmentation methods in OOD generalization performance.