Inferring Which Medical Treatments Work from Reports of Clinical Trials
This work addresses the burden on medical professionals and researchers in synthesizing evidence from unstructured clinical trial articles, though it is incremental as it focuses on creating a dataset and baseline models for a new task.
The authors tackled the problem of extracting actionable evidence from clinical trial reports by introducing a new task and corpus for inferring reported findings from full-text articles, and they demonstrated the difficulty of the task using various models, with results showing it remains challenging due to lengthy, technical inputs.
How do we know if a particular medical treatment actually works? Ideally one would consult all available evidence from relevant clinical trials. Unfortunately, such results are primarily disseminated in natural language scientific articles, imposing substantial burden on those trying to make sense of them. In this paper, we present a new task and corpus for making this unstructured evidence actionable. The task entails inferring reported findings from a full-text article describing a randomized controlled trial (RCT) with respect to a given intervention, comparator, and outcome of interest, e.g., inferring if an article provides evidence supporting the use of aspirin to reduce risk of stroke, as compared to placebo. We present a new corpus for this task comprising 10,000+ prompts coupled with full-text articles describing RCTs. Results using a suite of models --- ranging from heuristic (rule-based) approaches to attentive neural architectures --- demonstrate the difficulty of the task, which we believe largely owes to the lengthy, technical input texts. To facilitate further work on this important, challenging problem we make the corpus, documentation, a website and leaderboard, and code for baselines and evaluation available at http://evidence-inference.ebm-nlp.com/.