Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review
This work addresses cost reduction for legal and information retrieval professionals, but it is incremental as it builds on prior methods.
The paper tackled the problem of reducing manual document assessment costs in Technology Assisted Review by extending an existing stopping rule with classifier-derived information, achieving consistent performance improvements across multiple datasets.
Technology Assisted Review (TAR) stopping rules aim to reduce the cost of manually assessing documents for relevance by minimising the number of documents that need to be examined to ensure a desired level of recall. This paper extends an effective stopping rule using information derived from a text classifier that can be trained without the need for any additional annotation. Experiments on multiple data sets (CLEF e-Health, TREC Total Recall, TREC Legal and RCV1) showed that the proposed approach consistently improves performance and outperforms several alternative methods.