Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression
This work addresses the challenge of scaling multi-output regression for applications requiring accurate correlation modeling, though it appears incremental by building on existing expert-based approximations of Gaussian processes.
The paper tackles large-scale multi-output regression by developing a model that uses a mixture of single-output Gaussian process experts encoded via probabilistic circuits to capture correlations between output dimensions. The approach demonstrates improved performance over methods ignoring inter-output correlations on several datasets, as measured by negative log predictive density.
Inspired by recent advances in the field of expert-based approximations of Gaussian processes (GPs), we present an expert-based approach to large-scale multi-output regression using single-output GP experts. Employing a deeply structured mixture of single-output GPs encoded via a probabilistic circuit allows us to capture correlations between multiple output dimensions accurately. By recursively partitioning the covariate space and the output space, posterior inference in our model reduces to inference on single-output GP experts, which only need to be conditioned on a small subset of the observations. We show that inference can be performed exactly and efficiently in our model, that it can capture correlations between output dimensions and, hence, often outperforms approaches that do not incorporate inter-output correlations, as demonstrated on several data sets in terms of the negative log predictive density.