Auto-bidding in real-time auctions via Oracle Imitation Learning (OIL)
This addresses the challenge of efficient auto-bidding for advertisers in online advertising, offering a novel approach that shifts complexity from learning algorithms to optimization, but it is incremental as it builds on imitation learning and optimization techniques.
The paper tackles the problem of training auto-bidding agents in real-time auctions to maximize acquisitions under budget and CPA constraints, by proposing Oracle Imitation Learning (OIL) which uses an oracle solution as a training target, and demonstrates superior performance and sample efficiency compared to reinforcement learning methods.
Online advertising has become one of the most successful business models of the internet era. Impression opportunities are typically allocated through real-time auctions, where advertisers bid to secure advertisement slots. Deciding the best bid for an impression opportunity is challenging, due to the stochastic nature of user behavior and the variability of advertisement traffic over time. In this work, we propose a framework for training auto-bidding agents in multi-slot second-price auctions to maximize acquisitions (e.g., clicks, conversions) while adhering to budget and cost-per-acquisition (CPA) constraints. We exploit the insight that, after an advertisement campaign concludes, determining the optimal bids for each impression opportunity can be framed as a multiple-choice knapsack problem (MCKP) with a nonlinear objective. We propose an "oracle" algorithm that identifies a near-optimal combination of impression opportunities and advertisement slots, considering both past and future advertisement traffic data. This oracle solution serves as a training target for a student network which bids having access only to real-time information, a method we term Oracle Imitation Learning (OIL). Through numerical experiments, we demonstrate that OIL achieves superior performance compared to both online and offline reinforcement learning algorithms, offering improved sample efficiency. Notably, OIL shifts the complexity of training auto-bidding agents from crafting sophisticated learning algorithms to solving a nonlinear constrained optimization problem efficiently.