High Dimensional Data Enrichment: Interpretable, Fast, and Data-Efficient
This work addresses the problem of efficient and interpretable multi-task learning for high-dimensional data, presenting a foundational analysis but with incremental methodological contributions.
The paper tackles multi-task learning in high-dimensional settings by introducing an estimator for data enrichment/sharing with connected linear regressions, showing that the common parameter benefits from all pooled samples and providing non-asymptotic error bounds and a fast iterative algorithm.
We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational properties for the problem of multiple connected linear regressions known as Data Enrichment/Sharing. The between-tasks connections are captured by a cross-tasks \emph{common parameter}, which gets refined by per-task \emph{individual parameters}. Any convex function, e.g., norm, can characterize the structure of both common and individual parameters. We delineate the sample complexity of our estimator and provide a high probability non-asymptotic bound for estimation error of all parameters under a geometric condition. We show that the recovery of the common parameter benefits from \emph{all} of the pooled samples. We propose an iterative estimation algorithm with a geometric convergence rate and supplement our theoretical analysis with experiments on synthetic data. Overall, we present a first thorough statistical and computational analysis of inference in the data-sharing model.