Modeling Document Interactions for Learning to Rank with Regularized Self-Attention
This work addresses ranking tasks in information retrieval, offering a novel regularization approach for self-attention, though it appears incremental as it builds on existing neural architectures.
The paper tackled the problem of learning to rank by modeling document interactions with self-attention networks, finding that standard self-attention offered little benefit, and proposed regularization terms that significantly outperformed existing methods on LETOR datasets.
Learning to rank is an important task that has been successfully deployed in many real-world information retrieval systems. Most existing methods compute relevance judgments of documents independently, without holistically considering the entire set of competing documents. In this paper, we explore modeling documents interactions with self-attention based neural networks. Although self-attention networks have achieved state-of-the-art results in many NLP tasks, we find empirically that self-attention provides little benefit over baseline neural learning to rank architecture. To improve the learning of self-attention weights, We propose simple yet effective regularization terms designed to model interactions between documents. Evaluations on publicly available Learning to Rank (LETOR) datasets show that training self-attention network with our proposed regularization terms can significantly outperform existing learning to rank methods.