Enhancing Clinical Information Extraction with Transferred Contextual Embeddings
This work addresses information extraction in clinical documents, showing incremental improvement for healthcare NLP applications.
The paper tackled the problem of applying BERT to clinical information extraction, achieving a new state-of-the-art macro-average F1 score of 0.438 on a nursing handover dataset, compared to 0.416 for previous best models.
The Bidirectional Encoder Representations from Transformers (BERT) model has achieved the state-of-the-art performance for many natural language processing (NLP) tasks. Yet, limited research has been contributed to studying its effectiveness when the target domain is shifted from the pre-training corpora, for example, for biomedical or clinical NLP applications. In this paper, we applied it to a widely studied a hospital information extraction (IE) task and analyzed its performance under the transfer learning setting. Our application became the new state-of-the-art result by a clear margin, compared with a range of existing IE models. Specifically, on this nursing handover data set, the macro-average F1 score from our model was 0.438, whilst the previous best deep learning models had 0.416. In conclusion, we showed that BERT based pre-training models can be transferred to health-related documents under mild conditions and with a proper fine-tuning process.