Multiple Output Regression with Latent Noise
This work addresses the problem of handling structured noise in high-dimensional data for researchers and practitioners in fields like metabolomics, genomics, and economics, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the challenge of structured noise in multiple-output regression by proposing a latent variable model that jointly models signal and noise, introducing a latent signal-to-noise ratio hyperparameter and an ordered infinite-dimensional shrinkage prior. The model matched or exceeded state-of-the-art performance in simulations and experiments across various datasets, outperforming standard independent noise and signal approaches.
In high-dimensional data, structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously, imposes a serious challenge for modeling, by masking the often weak signal. Therefore, (1) explaining away the structured noise in multiple-output regression is of paramount importance. Additionally, (2) assumptions about the correlation structure of the regression weights are needed. We note that both can be formulated in a natural way in a latent variable model, in which both the interesting signal and the noise are mediated through the same latent factors. Under this assumption, the signal model then borrows strength from the noise model by encouraging similar effects on correlated targets. We introduce a hyperparameter for the \emph{latent signal-to-noise ratio} which turns out to be important for modelling weak signals, and an ordered infinite-dimensional shrinkage prior that resolves the rotational unidentifiability in reduced-rank regression models. Simulations and prediction experiments with metabolite, gene expression, FMRI measurement, and macroeconomic time series data show that our model equals or exceeds the state-of-the-art performance and, in particular, outperforms the standard approach of assuming independent noise and signal models.