Production Ranking Systems: A Review
This is an incremental review aimed at familiarizing a general audience with ranking systems at scale, addressing the multi-billion dollar problem of ranking for users in production environments.
The paper tackles the challenge of designing production ranking systems that balance effective machine learning models with real-time user response, describing them as layered systems comprising data processing, representation learning, candidate selection, and online inference. It provides an overview of these systems, their algorithms, and the challenges of a layered approach.
The problem of ranking is a multi-billion dollar problem. In this paper we present an overview of several production quality ranking systems. We show that due to conflicting goals of employing the most effective machine learning models and responding to users in real time, ranking systems have evolved into a system of systems, where each subsystem can be viewed as a component layer. We view these layers as being data processing, representation learning, candidate selection and online inference. Each layer employs different algorithms and tools, with every end-to-end ranking system spanning multiple architectures. Our goal is to familiarize the general audience with a working knowledge of ranking at scale, the tools and algorithms employed and the challenges introduced by adopting a layered approach.