Online optimization and regret guarantees for non-additive long-term constraints
This work addresses non-stationary constraints in online optimization, which is incremental for applications like display advertising.
The paper tackles online optimization with non-additive long-term constraints, such as in online display advertising, by proposing a primal-dual algorithm that achieves dynamic cumulative regret guarantees, with experiments showing vanishing regret on real-world data.
We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game. The resulting formulation enables us to deal with non-stationary and/or long-term constraints , which arise, for example, in online display advertising problems. We propose an on-line primal-dual algorithm for which we obtain dynamic cumulative regret guarantees. They depend on the convexity and the smoothness of the non-additive penalty, as well as terms capturing the smoothness with which the residuals of the non-stationary and long-term constraints vary over the rounds. We conduct experiments on synthetic data to illustrate the benefits of the non-additive penalty and show vanishing regret convergence on live traffic data collected by a display advertising platform in production.