Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents
This addresses the challenge of low recall in relation extraction for populating long lists from long texts, which is incremental as it builds on existing LLM and retrieval methods.
The paper tackles the problem of extracting long lists of object entities from long documents with high recall, presenting the L3X method that uses retrieval-augmented LLMs and precision-oriented scrutinization, which outperforms LLM-only generations by a substantial margin.
Methods for relation extraction from text mostly focus on high precision, at the cost of limited recall. High recall is crucial, though, to populate long lists of object entities that stand in a specific relation with a given subject. Cues for relevant objects can be spread across many passages in long texts. This poses the challenge of extracting long lists from long texts. We present the L3X method which tackles the problem in two stages: (1) recall-oriented generation using a large language model (LLM) with judicious techniques for retrieval augmentation, and (2) precision-oriented scrutinization to validate or prune candidates. Our L3X method outperforms LLM-only generations by a substantial margin.