Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks
This work addresses the problem of improving ranking accuracy in information retrieval systems, particularly for email search engines, by introducing a novel multivariate approach that is not incremental but builds on existing methods.
The paper tackles the limitation of univariate scoring functions in learning-to-rank by proposing groupwise scoring functions (GSFs), which determine document relevance scores jointly based on multiple documents in a list, and demonstrates significant performance improvements in evaluations using a commercial email search engine and a public benchmark dataset, especially with sparse textual features.
While in a classification or a regression setting a label or a value is assigned to each individual document, in a ranking setting we determine the relevance ordering of the entire input document list. This difference leads to the notion of relative relevance between documents in ranking. The majority of the existing learning-to-rank algorithms model such relativity at the loss level using pairwise or listwise loss functions. However, they are restricted to univariate scoring functions, i.e., the relevance score of a document is computed based on the document itself, regardless of other documents in the list. To overcome this limitation, we propose a new framework for multivariate scoring functions, in which the relevance score of a document is determined jointly by multiple documents in the list. We refer to this framework as GSFs -- groupwise scoring functions. We learn GSFs with a deep neural network architecture, and demonstrate that several representative learning-to-rank algorithms can be modeled as special cases in our framework. We conduct evaluation using click logs from one of the largest commercial email search engines, as well as a public benchmark dataset. In both cases, GSFs lead to significant performance improvements, especially in the presence of sparse textual features.