Representation Retrieval Learning for Heterogeneous Data Integration
This addresses data integration problems for predictive modeling in big data contexts, but it appears incremental as it builds on multi-task learning with partial sharing.
The paper tackles the challenge of integrating heterogeneous, multi-modal datasets with issues like covariate shift and missing modalities by proposing a Representation Retrieval (R^2) framework that combines representation learning with sparsity-induced models, achieving superior performance over existing methods in simulations and real-world applications.
In the era of big data, large-scale, multi-modal datasets are increasingly ubiquitous, offering unprecedented opportunities for predictive modeling and scientific discovery. However, these datasets often exhibit complex heterogeneity, such as covariate shift, posterior drift, and missing modalities, that can hinder the accuracy of existing prediction algorithms. To address these challenges, we propose a novel Representation Retrieval ($R^2$) framework, which integrates a representation learning module (the representer) with a sparsity-induced machine learning model (the learner). Moreover, we introduce the notion of "integrativeness" for representers, characterized by the effective data sources used in learning representers, and propose a Selective Integration Penalty (SIP) to explicitly improve the property. Theoretically, we demonstrate that the $R^2$ framework relaxes the conventional full-sharing assumption in multi-task learning, allowing for partially shared structures, and that SIP can improve the convergence rate of the excess risk bound. Extensive simulation studies validate the empirical performance of our framework, and applications to two real-world datasets further confirm its superiority over existing approaches.