What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns
This work supports prompt engineering and human-centric model behavior research by providing a more efficient tool for users to analyze LLM outputs, though it is incremental as it builds on existing data mining techniques.
The paper tackles the challenge of evaluating large language model outputs by proposing Spotlight, a method that automatically distinguishes random variations from systematic differences using token patterns, and demonstrates its effectiveness through benchmarks and user studies, showing it helps users identify relevant differences like gender or culture biases.
Prompt engineering for large language models is challenging, as even small prompt perturbations or model changes can significantly impact the generated output texts. Existing evaluation methods of LLM outputs, either automated metrics or human evaluation, have limitations, such as providing limited insights or being labor-intensive. We propose Spotlight, a new approach that combines both automation and human analysis. Based on data mining techniques, we automatically distinguish between random (decoding) variations and systematic differences in language model outputs. This process provides token patterns that describe the systematic differences and guide the user in manually analyzing the effects of their prompts and changes in models efficiently. We create three benchmarks to quantitatively test the reliability of token pattern extraction methods and demonstrate that our approach provides new insights into established prompt data. From a human-centric perspective, through demonstration studies and a user study, we show that our token pattern approach helps users understand the systematic differences of language model outputs. We are further able to discover relevant differences caused by prompt and model changes (e.g. related to gender or culture), thus supporting the prompt engineering process and human-centric model behavior research.