RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification
This work addresses the need for accurate radiology report classification for quality control and disease monitoring in healthcare, representing an incremental advance by enhancing existing transformer methods with specialized augmentations.
The paper tackled the problem of extracting pathologic findings from unstructured radiology reports by developing RadBERT-CL, a model that uses factually-aware contrastive learning to handle clinical facts and uncertain statements, resulting in a 6-11% performance improvement over existing transformers when few labeled data are available.
Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule-based systems or transformer-based pre-trained model fine-tuning, but could not take the factual and uncertain information into consideration, and therefore generate false-positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT-CL, which fuses these information into BlueBert via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label report classification. We illustrate that when few labeled data are available, RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by significantly larger margins (6-11%). We also show that the representations learned by RadBERT-CL can capture critical medical information in the latent space.