Enhancing Spectral Embedding through Robust and Flexible Knowledge Transfer in Electronic Health Records
For researchers working with high-dimensional, small-sample electronic health records in rare diseases, this method provides more robust embeddings by flexibly transferring knowledge from larger populations.
The paper proposes a spectral-based unsupervised representation learning framework that uses a knowledge matrix from a broader population to improve embeddings for rare disease cohorts. The method outperforms competing approaches in simulations and real-world multiple sclerosis data, especially when shared signals are weak and partially aligned.
We propose a spectral-based, unsupervised representation learning framework to derive low-dimensional embeddings for clinical concepts and patients in rare disease cohorts from electronic health records, where data are high-dimensional but sample sizes are limited. To overcome this challenge, we incorporate a knowledge matrix extracted from a broader population that shares a partially overlapping subspace with the rare-disease cohort. Our method departs from existing approaches by relaxing restrictive one-to-one signal-alignment assumptions between the latent data matrix and knowledge matrix, allowing more flexible and realistic forms of structured sharing. We introduce a novel two-step spectral embedding procedure: first, we identify and remove irrelevant components from the knowledge matrix; then, we apply a projection-based method to separately recover shared and heterogeneous components. Simulations and an analysis of a real-world multiple sclerosis cohort show that the proposed method outperforms competing approaches, particularly in challenging scenarios where shared signals are weak and only partially aligned, as is common in rare-disease data.