Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems
This work addresses a core theoretical gap for practitioners using structured learning methods in combinatorial optimization, though it is incremental as it builds on existing surrogate policy frameworks.
The paper tackles the challenge of training surrogate policies for combinatorial optimization problems by analyzing smoothed policies that add random perturbations to make the empirical risk differentiable, resulting in a generalization bound that decomposes excess risk into perturbation bias, statistical error, and optimization error.
A recent line of structured learning methods has advanced the practical state-of-the-art for combinatorial optimization problems with complex, application-specific objectives. These approaches learn policies that couple a statistical model with a tractable surrogate combinatorial optimization oracle, so as to exploit the distribution of problem instances instead of solving each instance independently. A core obstacle is that the empirical risk is then piecewise constant in the model parameters. This hinders gradient-based optimization and only few theoretical guarantees have been provided so far. We address this issue by analyzing smoothed (perturbed) policies: adding controlled random perturbations to the direction used by the linear oracle yields a differentiable surrogate risk and improves generalization. Our main contribution is a generalization bound that decomposes the excess risk into perturbation bias, statistical estimation error, and optimization error. The analysis hinges on a new Uniform Weak (UW) property capturing the geometric interaction between the statistical model and the normal fan of the feasible polytope; we show it holds under mild assumptions, and automatically when a minimal baseline perturbation is present. The framework covers, in particular, contextual stochastic optimization. We illustrate the scope of the results on applications such as stochastic vehicle scheduling, highlighting how smoothing enables both tractable training and controlled generalization.