Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes
This work addresses the challenge of leveraging unstructured clinical notes for predictive modeling in healthcare, offering an incremental improvement over existing approaches.
The authors tackled the problem of classifying patient discharge diagnoses from heterogeneous clinical notes by proposing a pretrained hierarchical recurrent neural network model, which improved performance on the MIMIC-III dataset compared to baseline methods that treat notes as unordered terms or skip pretraining.
Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations. Using these notes in predictive modeling has traditionally required preprocessing (e.g. taking frequent terms or topic modeling) that removes much of the richness of the source data. We propose a pretrained hierarchical recurrent neural network model that parses minimally processed clinical notes in an intuitive fashion, and show that it improves performance for discharge diagnosis classification tasks on the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, compared to models that treat the notes as an unordered collection of terms or that conduct no pretraining. We also apply an attribution technique to examples to identify the words that the model uses to make its prediction, and show the importance of the words' nearby context.