Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains
This addresses a domain-specific problem for multi-task learning in scenarios with heterogeneous input features, representing an incremental advancement.
The paper tackles the problem of multi-task Gaussian processes being limited to tasks with the same input domain by proposing a heterogeneous stochastic variational linear model of coregionalization (HSVLMC) for tasks with varied input domains, achieving superiority over existing models in diverse cases and a practical steam turbine exhaust problem.
Multi-task Gaussian process (MTGP) is a well-known non-parametric Bayesian model for learning correlated tasks effectively by transferring knowledge across tasks. But current MTGPs are usually limited to the multi-task scenario defined in the same input domain, leaving no space for tackling the heterogeneous case, i.e., the features of input domains vary over tasks. To this end, this paper presents a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) model for simultaneously learning the tasks with varied input domains. Particularly, we develop the stochastic variational framework with Bayesian calibration that (i) takes into account the effect of dimensionality reduction raised by domain mappings in order to achieve effective input alignment; and (ii) employs a residual modeling strategy to leverage the inductive bias brought by prior domain mappings for better model inference. Finally, the superiority of the proposed model against existing LMC models has been extensively verified on diverse heterogeneous multi-task cases and a practical multi-fidelity steam turbine exhaust problem.