SuPreME: A Supervised Pre-training Framework for Multimodal ECG Representation Learning
This work addresses the challenge of fine-grained clinical semantics in ECG analysis for healthcare applications, offering a zero-shot classification approach that reduces the need for task-specific fine-tuning.
The paper tackled the problem of learning ECG representations without extensive labeled data by proposing SuPreME, a supervised pre-training framework that uses LLMs to extract structured diagnostic labels from ECG reports, achieving a zero-shot AUC of 77.20% and outperforming state-of-the-art methods by 4.98% on six datasets covering 106 cardiac conditions.
Cardiovascular diseases are a leading cause of death and disability worldwide. Electrocardiogram (ECG) is critical for diagnosing and monitoring cardiac health, but obtaining large-scale annotated ECG datasets is labor-intensive and time-consuming. Recent ECG Self-Supervised Learning (eSSL) methods mitigate this by learning features without extensive labels but fail to capture fine-grained clinical semantics and require extensive task-specific fine-tuning. To address these challenges, we propose $\textbf{SuPreME}$, a $\textbf{Su}$pervised $\textbf{Pre}$-training framework for $\textbf{M}$ultimodal $\textbf{E}$CG representation learning. SuPreME is pre-trained using structured diagnostic labels derived from ECG report entities through a one-time offline extraction with Large Language Models (LLMs), which help denoise, standardize cardiac concepts, and improve clinical representation learning. By fusing ECG signals with textual cardiac queries instead of fixed labels, SuPreME enables zero-shot classification of unseen conditions without further fine-tuning. We evaluate SuPreME on six downstream datasets covering 106 cardiac conditions, achieving superior zero-shot AUC performance of $77.20\%$, surpassing state-of-the-art eSSLs by $4.98\%$. Results demonstrate SuPreME's effectiveness in leveraging structured, clinically relevant knowledge for high-quality ECG representations.