The Impact of Batch Learning in Stochastic Bandits
This work addresses batch learning limitations in stochastic bandits for applications like recommender systems, but it is incremental as it builds on prior bandit research with a batch-centric focus.
The paper tackles the problem of batched bandits, motivated by practical constraints in recommender systems and e-commerce, by analyzing regret bounds for batch learning scenarios and showing that batch impact relates to online behavior, with empirical validation on optimal batch size.
We consider a special case of bandit problems, namely batched bandits. Motivated by natural restrictions of recommender systems and e-commerce platforms, we assume that a learning agent observes responses batched in groups over a certain time period. Unlike previous work, we consider a more practically relevant batch-centric scenario of batch learning. We provide a policy-agnostic regret analysis and demonstrate upper and lower bounds for the regret of a candidate policy. Our main theoretical results show that the impact of batch learning can be measured in terms of online behavior. Finally, we demonstrate the consistency of theoretical results by conducting empirical experiments and reflect on the optimal batch size choice.