Discriminative Representation Learning for Clinical Prediction
This work addresses clinical prediction in healthcare by challenging the need for large-scale self-supervised pretraining, offering a more efficient and effective approach for outcome-driven domains.
The paper tackles the problem of clinical prediction by proposing a supervised deep learning framework that directly aligns representations with clinical outcomes, outperforming self-supervised pretraining baselines in tasks like mortality and readmission prediction with improved discrimination, calibration, and sample efficiency.
Foundation models in healthcare have largely adopted self supervised pretraining objectives inherited from natural language processing and computer vision, emphasizing reconstruction and large scale representation learning prior to downstream adaptation. We revisit this paradigm in outcome centric clinical prediction settings and argue that, when high quality supervision is available, direct outcome alignment may provide a stronger inductive bias than generative pretraining. We propose a supervised deep learning framework that explicitly shapes representation geometry by maximizing inter class separation relative to within class variance, thereby concentrating model capacity along clinically meaningful axes. Across multiple longitudinal electronic health record tasks, including mortality and readmission prediction, our approach consistently outperforms masked, autoregressive, and contrastive pretraining baselines under matched model capacity. The proposed method improves discrimination, calibration, and sample efficiency, while simplifying the training pipeline to a single stage optimization. These findings suggest that in low entropy, outcome driven healthcare domains, supervision can act as the statistically optimal driver of representation learning, challenging the assumption that large scale self supervised pretraining is a prerequisite for strong clinical performance.