Integrating Text and Time-Series into (Large) Language Models to Predict Medical Outcomes
This work addresses the challenge of integrating diverse data types in medical AI for clinicians, though it appears incremental as it builds on existing LLM and prompt optimization techniques.
The paper tackled the problem of adapting large language models (LLMs) to handle clinical classification tasks involving both text and structured time-series data, achieving performance comparable to specialized multimodal systems with reduced complexity and improved adaptability.
Large language models (LLMs) excel at text generation, but their ability to handle clinical classification tasks involving structured data, such as time series, remains underexplored. In this work, we adapt instruction-tuned LLMs using DSPy-based prompt optimization to process clinical notes and structured EHR inputs jointly. Our results show that this approach achieves performance on par with specialized multimodal systems while requiring less complexity and offering greater adaptability across tasks.