Learning To Rank Diversely At Airbnb
This work solves the problem of improving search result diversity for Airbnb's marketplace, offering a practical solution for large-scale production systems, though it is incremental as it builds on existing learning to rank frameworks.
The paper tackles the problem of ranking listings in Airbnb's search results by addressing the false assumption that booking probabilities are independent, showing that explicitly reducing similarities between listings to diversify results leads to strong positive impact in online A/B tests.
Airbnb is a two-sided marketplace, bringing together hosts who own listings for rent, with prospective guests from around the globe. Applying neural network-based learning to rank techniques has led to significant improvements in matching guests with hosts. These improvements in ranking were driven by a core strategy: order the listings by their estimated booking probabilities, then iterate on techniques to make these booking probability estimates more and more accurate. Embedded implicitly in this strategy was an assumption that the booking probability of a listing could be determined independently of other listings in search results. In this paper we discuss how this assumption, pervasive throughout the commonly-used learning to rank frameworks, is false. We provide a theoretical foundation correcting this assumption, followed by efficient neural network architectures based on the theory. Explicitly accounting for possible similarities between listings, and reducing them to diversify the search results generated strong positive impact. We discuss these metric wins as part of the online A/B tests of the theory. Our method provides a practical way to diversify search results for large-scale production ranking systems.