CARE: Extracting Experimental Findings From Clinical Literature
This work addresses the challenge of extracting nuanced experimental findings from biomedical literature, which is incremental as it builds on prior limited datasets by providing a more comprehensive schema and benchmark.
The authors tackled the problem of extracting fine-grained experimental findings from clinical literature by introducing CARE, a new information extraction dataset with a unified annotation schema that captures complex phenomena like discontinuous entities and n-ary relations, and they showed that state-of-the-art models like GPT4 struggle on this dataset.
Extracting fine-grained experimental findings from literature can provide dramatic utility for scientific applications. Prior work has developed annotation schemas and datasets for limited aspects of this problem, failing to capture the real-world complexity and nuance required. Focusing on biomedicine, this work presents CARE -- a new IE dataset for the task of extracting clinical findings. We develop a new annotation schema capturing fine-grained findings as n-ary relations between entities and attributes, which unifies phenomena challenging for current IE systems such as discontinuous entity spans, nested relations, variable arity n-ary relations and numeric results in a single schema. We collect extensive annotations for 700 abstracts from two sources: clinical trials and case reports. We also demonstrate the generalizability of our schema to the computer science and materials science domains. We benchmark state-of-the-art IE systems on CARE, showing that even models such as GPT4 struggle. We release our resources to advance research on extracting and aggregating literature findings.