Towards Foundation Models for Critical Care Time Series
This work addresses the challenge of distribution shifts in critical care data for researchers and practitioners, but it is incremental as it focuses on dataset creation rather than novel model development.
The paper tackled the problem of limited large-scale modeling for critical care time series data by creating a harmonized dataset that addresses distribution shifts from varying treatment policies, resulting in the first large-scale collection including core treatment variables to support transfer learning across hospitals.
Notable progress has been made in generalist medical large language models across various healthcare areas. However, large-scale modeling of in-hospital time series data - such as vital signs, lab results, and treatments in critical care - remains underexplored. Existing datasets are relatively small, but combining them can enhance patient diversity and improve model robustness. To effectively utilize these combined datasets for large-scale modeling, it is essential to address the distribution shifts caused by varying treatment policies, necessitating the harmonization of treatment variables across the different datasets. This work aims to establish a foundation for training large-scale multi-variate time series models on critical care data and to provide a benchmark for machine learning models in transfer learning across hospitals to study and address distribution shift challenges. We introduce a harmonized dataset for sequence modeling and transfer learning research, representing the first large-scale collection to include core treatment variables. Future plans involve expanding this dataset to support further advancements in transfer learning and the development of scalable, generalizable models for critical healthcare applications.