Paging Dr. GPT: Extracting Information from Clinical Notes to Enhance Patient Predictions
This addresses the challenge of underutilized unstructured text data in healthcare for clinicians and researchers, though it is incremental as it applies an existing LLM method to a new domain.
The study tackled the problem of extracting information from unstructured clinical notes to improve patient mortality predictions, showing that GPT-based models alone outperform standard tabular models and combining both sources increases AUC by 5.1 percentage points and positive predictive value by 29.9% for high-risk patients.
There is a long history of building predictive models in healthcare using tabular data from electronic medical records. However, these models fail to extract the information found in unstructured clinical notes, which document diagnosis, treatment, progress, medications, and care plans. In this study, we investigate how answers generated by GPT-4o-mini (ChatGPT) to simple clinical questions about patients, when given access to the patient's discharge summary, can support patient-level mortality prediction. Using data from 14,011 first-time admissions to the Coronary Care or Cardiovascular Intensive Care Units in the MIMIC-IV Note dataset, we implement a transparent framework that uses GPT responses as input features in logistic regression models. Our findings demonstrate that GPT-based models alone can outperform models trained on standard tabular data, and that combining both sources of information yields even greater predictive power, increasing AUC by an average of 5.1 percentage points and increasing positive predictive value by 29.9 percent for the highest-risk decile. These results highlight the value of integrating large language models (LLMs) into clinical prediction tasks and underscore the broader potential for using LLMs in any domain where unstructured text data remains an underutilized resource.