Inductive Bias Extraction and Matching for LLM Prompts
This addresses prompt engineering challenges for LLM users, but it is incremental as it builds on existing methods for optimizing prompts.
The paper tackles the problem of LLM sensitivity to prompt wording by extracting and matching the model's inductive bias, resulting in improvements of up to 19% in classification Likert ratings and up to 27% in ranking Likert ratings.
The active research topic of prompt engineering makes it evident that LLMs are sensitive to small changes in prompt wording. A portion of this can be ascribed to the inductive bias that is present in the LLM. By using an LLM's output as a portion of its prompt, we can more easily create satisfactory wording for prompts. This has the effect of creating a prompt that matches the inductive bias in model. Empirically, we show that using this Inductive Bias Extraction and Matching strategy improves LLM Likert ratings used for classification by up to 19% and LLM Likert ratings used for ranking by up to 27%.