Time-Aware Attention for Enhanced Electronic Health Records Modeling
This work addresses the challenge of complex temporal patterns in EHRs for healthcare decision-making, representing an incremental improvement by combining existing Transformer and LLM techniques with novel temporal modeling.
The paper tackled the problem of modeling Electronic Health Records (EHRs) by addressing irregular time intervals between clinical events, proposing TALE-EHR with a time-aware attention mechanism and LLM-derived embeddings, which outperformed state-of-the-art baselines on disease progression forecasting tasks using MIMIC-IV and PIC datasets.
Electronic Health Records (EHR) contain valuable clinical information for predicting patient outcomes and guiding healthcare decisions. However, effectively modeling Electronic Health Records (EHRs) requires addressing data heterogeneity and complex temporal patterns. Standard approaches often struggle with irregular time intervals between clinical events. We propose TALE-EHR, a Transformer-based framework featuring a novel time-aware attention mechanism that explicitly models continuous temporal gaps to capture fine-grained sequence dynamics. To complement this temporal modeling with robust semantics, TALE-EHR leverages embeddings derived from standardized code descriptions using a pre-trained Large Language Model (LLM), providing a strong foundation for understanding clinical concepts. Experiments on the MIMIC-IV and PIC dataset demonstrate that our approach outperforms state-of-the-art baselines on tasks such as disease progression forecasting. TALE-EHR underscores the benefit of integrating explicit, continuous temporal modeling with strong semantic representations provides a powerful solution for advancing EHR analysis.