Listwise Learning to Rank with Deep Q-Networks
This work addresses ranking accuracy for information retrieval systems, though it is incremental as it builds on existing deep Q-learning methods with a specific dataset application.
The paper tackles the problem of ranking documents for queries by proposing DeepQRank, a deep Q-learning agent, which achieves a state-of-the-art NDCG@1 score of 0.5075 on the LETOR dataset, narrowly outperforming SVMRank (0.4958).
Learning to Rank is the problem involved with ranking a sequence of documents based on their relevance to a given query. Deep Q-Learning has been shown to be a useful method for training an agent in sequential decision making. In this paper, we show that DeepQRank, our deep q-learning to rank agent, demonstrates performance that can be considered state-of-the-art. Though less computationally efficient than a supervised learning approach such as linear regression, our agent has fewer limitations in terms of which format of data it can use for training and evaluation. We run our algorithm against Microsoft's LETOR listwise dataset and achieve an NDCG@1 (ranking accuracy in the range [0,1]) of 0.5075, narrowly beating out the leading supervised learning model, SVMRank (0.4958).