A pragmatic policy learning approach to account for users' fatigue in repeated auctions
This addresses a specific issue for online advertisers by mitigating the cost of impatience in bidding strategies, though it appears incremental as it builds on existing ML predictions.
The paper tackles the problem of user fatigue in repeated online ad auctions, where showing more banners reduces their marginal value, and demonstrates that using policy learning to account for future impacts improves business metrics.
Online advertising banners are sold in real-time through auctions.Typically, the more banners a user is shown, the smaller the marginalvalue of the next banner for this user is. This fact can be detected bybasic ML models, that can be used to predict how previously won auctionsdecrease the current opportunity value. However, learning is not enough toproduce a bid that correctly accounts for how winning the current auctionimpacts the future values. Indeed, a policy that uses this prediction tomaximize the expected payoff of the current auction could be dubbedimpatient because such policy does not fully account for the repeatednature of the auctions. Under this perspective, it seems that most biddersin the literature are impatient. Unsurprisingly, impatience induces a cost.We provide two empirical arguments for the importance of this cost ofimpatience. First, an offline counterfactual analysis and, second, a notablebusiness metrics improvement by mitigating the cost of impatience withpolicy learning