A Regression Framework for Understanding Prompt Component Impact on LLM Performance
For developers and decision-makers using LLMs, this provides a granular tool to understand prompt influence, though the method is an incremental extension of existing XAI techniques.
The paper introduces a regression-based framework to analyze how specific prompt features affect LLM performance, applied to arithmetic tasks on Mistral-7B and GPT-OSS-20B. The models' performance variation is explained by 72% and 77% respectively, revealing that misinformation in prompts harms performance while positive examples have mixed effects.
As large language models (LLMs) continue to improve and see further integration into software systems, so does the need to understand the conditions in which they will perform. We contribute a statistical framework for understanding the impact of specific prompt features on LLM performance. The approach extends previous explainable artificial intelligence (XAI) methods specifically to inspect LLMs by fitting regression models relating portions of the prompt to LLM evaluation. We apply our method to compare how two open-source models, Mistral-7B and GPT-OSS-20B, leverage the prompt to perform a simple arithmetic problem. Regression models of individual prompt portions explain 72% and 77% of variation in model performances, respectively. We find misinformation in the form of incorrect example query-answer pairs impedes both models from solving the arithmetic query, though positive examples do not find significant variability in the impact of positive and negative instructions - these prompts have contradictory effects on model performance. The framework serves as a tool for decision makers in critical scenarios to gain granular insight into how the prompt influences an LLM to solve a task.