LILO: Bayesian Optimization with Interactive Natural Language Feedback
This provides a more natural interface for decision-makers in optimization tasks where feedback is nuanced or subjective, though it is an incremental improvement over existing preferential BO methods.
The paper tackles the problem of incorporating complex natural language feedback into Bayesian optimization by proposing a language-in-the-loop framework that uses LLMs to convert unstructured text into scalar utilities, outperforming conventional baselines and LLM-only optimizers in feedback-limited settings.
For many real-world applications, feedback is essential in translating complex, nuanced, or subjective goals into quantifiable optimization objectives. We propose a language-in-the-loop framework that uses a large language model (LLM) to convert unstructured feedback in the form of natural language into scalar utilities to conduct BO over a numeric search space. Unlike preferential BO, which only accepts restricted feedback formats and requires customized models for each domain-specific problem, our approach leverages LLMs to turn varied types of textual feedback into consistent utility signals and to easily include flexible user priors without manual kernel design. At the same time, our method maintains the sample efficiency and principled uncertainty quantification of BO. We show that this hybrid method not only provides a more natural interface to the decision maker but also outperforms conventional BO baselines and LLM-only optimizers, particularly in feedback-limited regimes.