Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
This work addresses a fundamental question in NLP for researchers and practitioners, but it is incremental as it builds on existing prompting literature without introducing new methods.
The study tackled the problem of understanding how prompting affects the internal representations of language models in zero-shot settings, finding that changes in representation quality do not consistently correlate with prompt relevance, challenging assumptions about prompt effectiveness.
Prompting is a common approach for leveraging LMs in zero-shot settings. However, the underlying mechanisms that enable LMs to perform diverse tasks without task-specific supervision remain poorly understood. Studying the relationship between prompting and the quality of internal representations can shed light on how pre-trained embeddings may support in-context task solving. In this empirical study, we conduct a series of probing experiments on prompt embeddings, analyzing various combinations of prompt templates for zero-shot classification. Our findings show that while prompting affects the quality of representations, these changes do not consistently correlate with the relevance of the prompts to the target task. This result challenges the assumption that more relevant prompts necessarily lead to better representations. We further analyze potential factors that may contribute to this unexpected behavior.