Extending Open Bandit Pipeline to Simulate Industry Challenges
This work provides a resource for e-commerce practitioners facing industry-specific bandit algorithm challenges, though it is incremental as it builds on an existing framework.
The paper tackles practical challenges in implementing bandit algorithms for e-commerce at Booking.com, such as off-policy optimization and delayed rewards, by extending the Open Bandit Pipeline framework with simulation components to help practitioners and researchers address these issues.
Bandit algorithms are often used in the e-commerce industry to train Machine Learning (ML) systems when pre-labeled data is unavailable. However, the industry setting poses various challenges that make implementing bandit algorithms in practice non-trivial. In this paper, we elaborate on the challenges of off-policy optimisation, delayed reward, concept drift, reward design, and business rules constraints that practitioners at Booking.com encounter when applying bandit algorithms. Our main contributions is an extension to the Open Bandit Pipeline (OBP) framework. We provide simulation components for some of the above-mentioned challenges to provide future practitioners, researchers, and educators with a resource to address challenges encountered in the e-commerce industry.