Elicitation Matters: How Prompts and Query Protocols Shape LLM Surrogates under Sparse Observations
For researchers using LLMs as surrogate models in low-data optimization, the paper demonstrates that elicitation protocol is a critical part of the surrogate specification, not a formatting detail.
The paper shows that LLM surrogates' predictions and uncertainties under sparse observations depend heavily on prompt text and query protocol, and introduces an uncertainty-alignment criterion to measure this. Structural prompts act as effective priors, querying methods induce different beliefs, and sequential evidence causes non-monotonic confidence updates, affecting downstream optimization regret.
Large language models are increasingly used as surrogate models for low-data optimization, but their optimizer-facing prediction and its uncertainty remain poorly understood. We study the surrogate belief elicited from an LLM under sparse observations, showing that it depends strongly on prompt text and query protocol. We introduce an uncertainty-alignment criterion that measures whether model uncertainty tracks residual ambiguity among sample-consistent functions. Across controlled inference tasks and Bayesian optimization studies, we find that structural prompts act as effective priors, POINTWISE and JOINT querying induce different beliefs, and sequential evidence leads to non-monotonic, order-sensitive confidence updates. These effects change downstream acquisition decisions and regret, showing that elicitation protocol is part of the LLM surrogate specification, not a formatting detail.