How are Prompts Different in Terms of Sensitivity?
This work addresses a gap in prompt engineering for researchers and practitioners in NLP, offering incremental insights into prompt analysis mechanisms.
The paper tackles the lack of systematic analysis of prompt effects in in-context learning by introducing sensitivity as an unsupervised proxy for model performance, showing a strong negative correlation with accuracy, and proposes sensitivity-aware decoding that improves performance when input information is scarce.
In-context learning (ICL) has become one of the most popular learning paradigms. While there is a growing body of literature focusing on prompt engineering, there is a lack of systematic analysis comparing the effects of prompts across different models and tasks. To address this gap, we present a comprehensive prompt analysis based on the sensitivity of a function. Our analysis reveals that sensitivity is an unsupervised proxy for model performance, as it exhibits a strong negative correlation with accuracy. We use gradient-based saliency scores to empirically demonstrate how different prompts affect the relevance of input tokens to the output, resulting in different levels of sensitivity. Furthermore, we introduce sensitivity-aware decoding which incorporates sensitivity estimation as a penalty term in the standard greedy decoding. We show that this approach is particularly helpful when information in the input is scarce. Our work provides a fresh perspective on the analysis of prompts, and contributes to a better understanding of the mechanism of ICL.