Latent Map Gaussian Processes for Mixed Variable Metamodeling
This work addresses the limitation of standard Gaussian processes in handling mixed data, which is a common problem for researchers and engineers in fields requiring metamodeling with diverse input types. It offers an incremental improvement to GP applicability.
The paper introduces Latent Map Gaussian Processes (LMGPs) to extend Gaussian processes (GPs) to mixed data with both quantitative and qualitative inputs. LMGPs learn a continuous, low-dimensional latent space for qualitative inputs, enabling their use in standard GP correlation functions. The authors demonstrate LMGPs' superior accuracy and versatility over state-of-the-art methods across various examples, including more efficient Bayesian optimization for compound compositions.
Gaussian processes (GPs) are ubiquitously used in sciences and engineering as metamodels. Standard GPs, however, can only handle numerical or quantitative variables. In this paper, we introduce latent map Gaussian processes (LMGPs) that inherit the attractive properties of GPs and are also applicable to mixed data which have both quantitative and qualitative inputs. The core idea behind LMGPs is to learn a continuous, low-dimensional latent space or manifold which encodes all qualitative inputs. To learn this manifold, we first assign a unique prior vector representation to each combination of qualitative inputs. We then use a low-rank linear map to project these priors on a manifold that characterizes the posterior representations. As the posteriors are quantitative, they can be directly used in any standard correlation function such as the Gaussian or Matern. Hence, the optimal map and the corresponding manifold, along with other hyperparameters of the correlation function, can be systematically learned via maximum likelihood estimation. Through a wide range of analytic and real-world examples, we demonstrate the advantages of LMGPs over state-of-the-art methods in terms of accuracy and versatility. In particular, we show that LMGPs can handle variable-length inputs, have an explainable neural network interpretation, and provide insights into how qualitative inputs affect the response or interact with each other. We also employ LMGPs in Bayesian optimization and illustrate that they can discover optimal compound compositions more efficiently than conventional methods that convert compositions to qualitative variables via manual featurization.