Bayesian task embedding for few-shot Bayesian optimization
This work addresses the challenge of data-efficient optimization for researchers and practitioners in fields like engineering or AI, though it is incremental as it builds on existing Bayesian optimization techniques.
The paper tackles the problem of Bayesian optimization across multiple systems with unknown interrelationships by introducing a metamodel that learns response surfaces using latent variables, achieving improved performance in zero-, one-, and few-shot settings compared to traditional methods that require more data.
We describe a method for Bayesian optimization by which one may incorporate data from multiple systems whose quantitative interrelationships are unknown a priori. All general (nonreal-valued) features of the systems are associated with continuous latent variables that enter as inputs into a single metamodel that simultaneously learns the response surfaces of all of the systems. Bayesian inference is used to determine appropriate beliefs regarding the latent variables. We explain how the resulting probabilistic metamodel may be used for Bayesian optimization tasks and demonstrate its implementation on a variety of synthetic and real-world examples, comparing its performance under zero-, one-, and few-shot settings against traditional Bayesian optimization, which usually requires substantially more data from the system of interest.