ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
This work addresses the problem of leveraging unstructured clinical data for hospital readmission prediction, which is incremental as it applies an existing method (BERT) to a new domain (clinical notes).
The paper tackled the underuse of clinical notes in healthcare by developing ClinicalBERT, a bidirectional transformer model that captures high-quality medical concept relationships and outperforms baselines in predicting 30-day hospital readmission using discharge summaries and ICU notes.
Clinical notes contain information about patients that goes beyond structured data like lab values and medications. However, clinical notes have been underused relative to structured data, because notes are high-dimensional and sparse. This work develops and evaluates representations of clinical notes using bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers high-quality relationships between medical concepts as judged by humans. ClinicalBert outperforms baselines on 30-day hospital readmission prediction using both discharge summaries and the first few days of notes in the intensive care unit. Code and model parameters are available.