Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction
This work addresses the challenge of analyzing clinical text for AI applications, but it is incremental as it applies an existing neural method (RoBERTa) to a specific dataset.
The paper tackled the problem of extracting temporal relations between clinical events from text, achieving a 0.0864 absolute F-measure improvement and a 24% relative error reduction compared to the previous state-of-the-art SVM model.
Background: Identifying relationships between clinical events and temporal expressions is a key challenge in meaningfully analyzing clinical text for use in advanced AI applications. While previous studies exist, the state-of-the-art performance has significant room for improvement. Methods: We studied several variants of BERT (Bidirectional Encoder Representations using Transformers) some involving clinical domain customization and the others involving improved architecture and/or training strategies. We evaluated these methods using a direct temporal relations dataset which is a semantically focused subset of the 2012 i2b2 temporal relations challenge dataset. Results: Our results show that RoBERTa, which employs better pre-training strategies including using 10x larger corpus, has improved overall F measure by 0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by 24% relative to the previous state-of-the-art performance achieved with an SVM (support vector machine) model. Conclusion: Modern contextual language modeling neural networks, pre-trained on a large corpus, achieve impressive performance even on highly-nuanced clinical temporal relation tasks.