Choppy: Cut Transformer For Ranked List Truncation
This addresses the critical but understudied issue of balancing result relevance with user processing costs in IR applications, though it is incremental as it builds on existing Transformer architectures.
The paper tackles the problem of optimally truncating ranked result lists in information retrieval by proposing Choppy, a Transformer-based model that directly optimizes user-defined IR metrics using only relevance scores, and shows improvements over recent state-of-the-art methods.
Work in information retrieval has traditionally focused on ranking and relevance: given a query, return some number of results ordered by relevance to the user. However, the problem of determining how many results to return, i.e. how to optimally truncate the ranked result list, has received less attention despite being of critical importance in a range of applications. Such truncation is a balancing act between the overall relevance, or usefulness of the results, with the user cost of processing more results. In this work, we propose Choppy, an assumption-free model based on the widely successful Transformer architecture, to the ranked list truncation problem. Needing nothing more than the relevance scores of the results, the model uses a powerful multi-head attention mechanism to directly optimize any user-defined IR metric. We show Choppy improves upon recent state-of-the-art methods.