Longformer for MS MARCO Document Re-ranking Task
This work addresses document re-ranking for information retrieval systems, but it is incremental as it adapts an existing model to a specific dataset without major innovations.
The authors tackled the MS MARCO document re-ranking task by applying Longformer, a BERT-like model for long documents, to improve performance in a two-step ranking system, achieving results that demonstrate its effectiveness in handling long documents.
Two step document ranking, where the initial retrieval is done by a classical information retrieval method, followed by neural re-ranking model, is the new standard. The best performance is achieved by using transformer-based models as re-rankers, e.g., BERT. We employ Longformer, a BERT-like model for long documents, on the MS MARCO document re-ranking task. The complete code used for training the model can be found on: https://github.com/isekulic/longformer-marco