Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction
This work addresses the problem of handling long patient trajectories with multi-modal data for clinicians, but it is incremental as it builds on existing LLM methods without major breakthroughs.
The study evaluated open-source LLMs, RAG variants, and CoT prompting on longitudinal clinical summarization and prediction using EHR data, finding that long context windows improved input integration but not clinical reasoning, and LLMs struggled with temporal progression and rare disease prediction.
Recent advances in large language models (LLMs) have shown potential in clinical text summarization, but their ability to handle long patient trajectories with multi-modal data spread across time remains underexplored. This study systematically evaluates several state-of-the-art open-source LLMs, their Retrieval Augmented Generation (RAG) variants and chain-of-thought (CoT) prompting on long-context clinical summarization and prediction. We examine their ability to synthesize structured and unstructured Electronic Health Records (EHR) data while reasoning over temporal coherence, by re-engineering existing tasks, including discharge summarization and diagnosis prediction from two publicly available EHR datasets. Our results indicate that long context windows improve input integration but do not consistently enhance clinical reasoning, and LLMs are still struggling with temporal progression and rare disease prediction. While RAG shows improvements in hallucination in some cases, it does not fully address these limitations. Our work fills the gap in long clinical text summarization, establishing a foundation for evaluating LLMs with multi-modal data and temporal reasoning.