Modelling Stopping Criteria for Search Results using Poisson Processes
This work addresses the need for efficient relevance assessment in large-scale document retrieval systems, presenting an incremental improvement over existing stopping criteria methods.
The paper tackles the problem of reducing manual evaluation effort in text retrieval by proposing a novel stopping criterion based on Poisson processes, allowing users to specify desired recall levels and confidence probabilities, and evaluates it on a public dataset with comparisons to previous techniques.
Text retrieval systems often return large sets of documents, particularly when applied to large collections. Stopping criteria can reduce the number of these documents that need to be manually evaluated for relevance by predicting when a suitable level of recall has been achieved. In this work, a novel method for determining a stopping criterion is proposed that models the rate at which relevant documents occur using a Poisson process. This method allows a user to specify both a minimum desired level of recall to achieve and a desired probability of having achieved it. We evaluate our method on a public dataset and compare it with previous techniques for determining stopping criteria.