Online Learning for Recommendations at Grubhub
This work addresses scalability and adaptability issues for large-scale recommendation platforms like Grubhub, though it is incremental as it modifies existing offline systems.
The paper tackles the problem of concept drift and high costs in offline recommender systems by transitioning to online learning, achieving up to 45x cost savings and a 20% increase in metrics at Grubhub.
We propose a method to easily modify existing offline Recommender Systems to run online using Transfer Learning. Online Learning for Recommender Systems has two main advantages: quality and scale. Like many Machine Learning algorithms in production if not regularly retrained will suffer from Concept Drift. A policy that is updated frequently online can adapt to drift faster than a batch system. This is especially true for user-interaction systems like recommenders where the underlying distribution can shift drastically to follow user behaviour. As a platform grows rapidly like Grubhub, the cost of running batch training jobs becomes material. A shift from stateless batch learning offline to stateful incremental learning online can recover, for example, at Grubhub, up to a 45x cost savings and a +20% metrics increase. There are a few challenges to overcome with the transition to online stateful learning, namely convergence, non-stationary embeddings and off-policy evaluation, which we explore from our experiences running this system in production.