Ensemble Multi-task Gaussian Process Regression with Multiple Latent Processes
This work addresses multi-task regression problems for domains like environmental monitoring, but it is incremental as it builds on existing GP frameworks.
The paper tackles multi-task learning with Gaussian Processes by proposing a model that decomposes covariance into latent processes, leading to improved regression performance on datasets like Swiss Jura and ATMS, with substantial gains over existing methods.
Multi-task/Multi-output learning seeks to exploit correlation among tasks to enhance performance over learning or solving each task independently. In this paper, we investigate this problem in the context of Gaussian Processes (GPs) and propose a new model which learns a mixture of latent processes by decomposing the covariance matrix into a sum of structured hidden components each of which is controlled by a latent GP over input features and a "weight" over tasks. From this sum structure, we propose a parallelizable parameter learning algorithm with a predetermined initialization for the "weights". We also notice that an ensemble parameter learning approach using mini-batches of training data not only reduces the computation complexity of learning but also improves the regression performance. We evaluate our model on two datasets, the smaller Swiss Jura dataset and another relatively larger ATMS dataset from NOAA. Substantial improvements are observed compared with established alternatives.