Traditional IR rivals neural models on the MS MARCO Document Ranking Leaderboard
This work demonstrates that traditional IR methods can still be competitive with, and even surpass, some neural approaches for document ranking, which is significant for researchers and practitioners in information retrieval.
A traditional IR system achieved an MRR@100 of 0.298 on the MS MARCO Document Ranking leaderboard, outperforming several neural models, including those using large pretrained Transformers for re-ranking.
This short document describes a traditional IR system that achieved MRR@100 equal to 0.298 on the MS MARCO Document Ranking leaderboard (on 2020-12-06). Although inferior to most BERT-based models, it outperformed several neural runs (as well as all non-neural ones), including two submissions that used a large pretrained Transformer model for re-ranking. We provide software and data to reproduce our results.