Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency
This addresses the bottleneck of needing annotated data for biomedical text analysis, specifically for radiologists and healthcare systems, though it is incremental as it applies an existing self-supervised method to a new domain.
The paper tackled the problem of identifying radiology reports requiring prompt communication by developing a self-supervised BERT model pre-trained on unlabeled reports, achieving a precision of 97.0%, recall of 93.3%, and F-measure of 95.1% on a test set, outperforming previous word2vec-based methods.
Machine learning methods have recently achieved high-performance in biomedical text analysis. However, a major bottleneck in the widespread application of these methods is obtaining the required large amounts of annotated training data, which is resource intensive and time consuming. Recent progress in self-supervised learning has shown promise in leveraging large text corpora without explicit annotations. In this work, we built a self-supervised contextual language representation model using BERT, a deep bidirectional transformer architecture, to identify radiology reports requiring prompt communication to the referring physicians. We pre-trained the BERT model on a large unlabeled corpus of radiology reports and used the resulting contextual representations in a final text classifier for communication urgency. Our model achieved a precision of 97.0%, recall of 93.3%, and F-measure of 95.1% on an independent test set in identifying radiology reports for prompt communication, and significantly outperformed the previous state-of-the-art model based on word2vec representations.