Missing-Data-Induced Phase Transitions in Spectral PLS for Multimodal Learning
This addresses the challenge of multimodal learning with missing data for researchers and practitioners, providing theoretical insights into performance limits, but it is incremental as it builds on existing spiked model analyses.
The paper tackled the problem of Partial Least Squares (PLS) failing to learn shared structure from multimodal data with missing entries, and found that PLS-SVD exhibits a sharp phase transition where below a critical signal-to-noise threshold, the leading singular vectors become uninformative, while above it they achieve nontrivial alignment with latent shared directions, with closed-form asymptotic overlap formulas.
Partial Least Squares (PLS) learns shared structure from paired data via the top singular vectors of the empirical cross-covariance (PLS-SVD), but multimodal datasets often have missing entries in both views. We study PLS-SVD under independent entry-wise missing-completely-at-random masking in a proportional high-dimensional spiked model. After appropriate normalization, the masked cross-covariance behaves like a spiked rectangular random matrix whose effective signal strength is attenuated by $\sqrtρ$, where $ρ$ is the joint entry retention probability. As a result, PLS-SVD exhibits a sharp BBP-type phase transition: below a critical signal-to-noise threshold the leading singular vectors are asymptotically uninformative, while above it they achieve nontrivial alignment with the latent shared directions, with closed-form asymptotic overlap formulas. Simulations and semi-synthetic multimodal experiments corroborate the predicted phase diagram and recovery curves across aspect ratios, signal strengths, and missingness levels.