Fast Slate Policy Optimization: Going Beyond Plackett-Luce
This work addresses a key challenge in search, information retrieval, and recommender systems by enabling efficient optimization for massive action spaces, representing an incremental improvement over existing methods.
The paper tackles the problem of optimizing large-scale decision systems for returning ordered lists (slates) under arbitrary reward functions, proposing a new policy class and learning algorithm that scales to millions of actions and outperforms the commonly used Plackett-Luce policy.
An increasingly important building block of large scale machine learning systems is based on returning slates; an ordered lists of items given a query. Applications of this technology include: search, information retrieval and recommender systems. When the action space is large, decision systems are restricted to a particular structure to complete online queries quickly. This paper addresses the optimization of these large scale decision systems given an arbitrary reward function. We cast this learning problem in a policy optimization framework and propose a new class of policies, born from a novel relaxation of decision functions. This results in a simple, yet efficient learning algorithm that scales to massive action spaces. We compare our method to the commonly adopted Plackett-Luce policy class and demonstrate the effectiveness of our approach on problems with action space sizes in the order of millions.