Rank1: Test-Time Compute for Reranking in Information Retrieval
This addresses the need for more efficient and explainable reranking models in search systems, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of improving information retrieval reranking by leveraging test-time compute, introducing Rank1 as the first model trained for this purpose, which achieves state-of-the-art performance on advanced reasoning datasets and works well out-of-distribution with explainable reasoning chains.
We introduce Rank1, the first reranking model trained to take advantage of test-time compute. Rank1 demonstrates the applicability within retrieval of using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for distillation in order to rapidly improve the performance of a smaller model. We gather and open-source a dataset of more than 600,000 examples of R1 reasoning traces from queries and passages in MS MARCO. Models trained on this dataset show: (1) state-of-the-art performance on advanced reasoning and instruction following datasets; (2) work remarkably well out of distribution due to the ability to respond to user-input prompts; and (3) have explainable reasoning chains that can be given to users or RAG-based systems. Further, we demonstrate that quantized versions of these models retain strong performance while using less compute/memory. Overall, Rank1 shows that test-time compute allows for a fundamentally new type of explainable and performant reranker model for search.