Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensions
This addresses the challenge of efficiently updating learning systems as they scale, offering a method for incremental improvements in matrix estimation tasks.
The paper tackles the problem of transfer learning for structured matrix estimation when both ambient dimensions and latent representations grow over time, proposing a framework that decomposes the target parameter into embedded source, low-rank innovations, and sparse edits. It achieves improved error rates when rank and sparsity increments are small, with demonstrated applications to Markov transition matrix and structured covariance estimation.
Learning systems often expand their ambient features or latent representations over time, embedding earlier representations into larger spaces with limited new latent structure. We study transfer learning for structured matrix estimation under simultaneous growth of the ambient dimension and the intrinsic representation, where a well-estimated source task is embedded as a subspace of a higher-dimensional target task. We propose a general transfer framework in which the target parameter decomposes into an embedded source component, low-dimensional low-rank innovations, and sparse edits, and develop an anchored alternating projection estimator that preserves transferred subspaces while estimating only low-dimensional innovations and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding strictly improved rates when rank and sparsity increments are small. We demonstrate the generality of the framework by applying it to two canonical problems. For Markov transition matrix estimation from a single trajectory, we derive end-to-end theoretical guarantees under dependent noise. For structured covariance estimation under enlarged dimensions, we provide complementary theoretical analysis in the appendix and empirically validate consistent transfer gains.