Coordinated Online Learning With Applications to Learning User Preferences
This work addresses the challenge of efficiently learning user preferences in dynamic environments like online marketplaces, though it appears incremental as it builds on existing online learning frameworks with coordination mechanisms.
The paper tackles the problem of online multi-task learning with related tasks arriving sequentially by proposing the COOL algorithm, which coordinates task-specific learners via weighted projections onto convex constraints, achieving a trade-off between coordination benefits and computational/communication costs with derived regret bounds.
We study an online multi-task learning setting, in which instances of related tasks arrive sequentially, and are handled by task-specific online learners. We consider an algorithmic framework to model the relationship of these tasks via a set of convex constraints. To exploit this relationship, we design a novel algorithm -- COOL -- for coordinating the individual online learners: Our key idea is to coordinate their parameters via weighted projections onto a convex set. By adjusting the rate and accuracy of the projection, the COOL algorithm allows for a trade-off between the benefit of coordination and the required computation/communication. We derive regret bounds for our approach and analyze how they are influenced by these trade-off factors. We apply our results on the application of learning users' preferences on the Airbnb marketplace with the goal of incentivizing users to explore under-reviewed apartments.