Exact and general decoupled solutions of the LMC Multitask Gaussian Process model
This work addresses computational bottlenecks in multitask Gaussian process modeling, offering a simpler and credible alternative to state-of-the-art models for researchers and practitioners in machine learning, though it is incremental as it extends prior decoupling results.
The paper tackles the computational inefficiency of the Linear Model of Co-regionalization (LMC) for multitask Gaussian processes by showing that an efficient exact computation is possible under mild noise conditions, introducing a projected LMC model with a marginal likelihood for optimization and demonstrating excellent performance on synthetic data compared to exact and approximate methods.
The Linear Model of Co-regionalization (LMC) is a very general model of multitask gaussian process for regression or classification. While its expressivity and conceptual simplicity are appealing, naive implementations have cubic complexity in the number of datapoints and number of tasks, making approximations mandatory for most applications. However, recent work has shown that under some conditions the latent processes of the model can be decoupled, leading to a complexity that is only linear in the number of said processes. We here extend these results, showing from the most general assumptions that the only condition necessary to an efficient exact computation of the LMC is a mild hypothesis on the noise model. We introduce a full parametrization of the resulting \emph{projected LMC} model, and an expression of the marginal likelihood enabling efficient optimization. We perform a parametric study on synthetic data to show the excellent performance of our approach, compared to an unrestricted exact LMC and approximations of the latter. Overall, the projected LMC appears as a credible and simpler alternative to state-of-the art models, which greatly facilitates some computations such as leave-one-out cross-validation and fantasization.