Building the EHR Foundation Model via Next Event Prediction
This work addresses the challenge of modeling sequential clinical events and temporal dependencies in EHRs for healthcare applications, representing an incremental improvement over existing methods.
The paper tackled the problem of capturing temporal dynamics in Electronic Health Records (EHRs) by proposing Next Event Prediction (NEP), a framework that enhances large language models' temporal reasoning through autoregressive fine-tuning on clinical event sequences, resulting in outperforming specialized EHR models by 4.6% AUROC and general-purpose LLMs by 7.2% C-index in temporal reasoning tasks.
Electronic Health Records (EHRs) contain rich temporal dynamics that conventional encoding approaches fail to adequately capture. While Large Language Models (LLMs) show promise for EHR modeling, they struggle to reason about sequential clinical events and temporal dependencies. We propose Next Event Prediction (NEP), a framework that enhances LLMs' temporal reasoning through autoregressive fine-tuning on clinical event sequences. By reformulating EHRs as timestamped event chains and predicting future medical events, NEP explicitly models disease progression patterns and causal relationships. Extensive evaluations across oncology survival prediction and clinical diagnosis tasks demonstrate NEP's superiority, outperforming specialized EHR models by 4.6% AUROC and general-purpose LLMs by 7.2% C-index in temporal reasoning tasks. Our analyses reveal dual benefits: state-of-the-art prediction accuracy combined with clinically interpretable attention patterns that align with known disease pathways.