When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations
This provides a theoretical foundation for understanding when these popular parameter-efficient fine-tuning techniques work, which is crucial for researchers and practitioners in NLP and AI, though it is incremental in building on existing empirical observations.
The paper tackles the theoretical limitations of context-based fine-tuning methods like prompting and prefix-tuning, showing that despite their empirical success, they are potentially less expressive than full fine-tuning because they cannot change relative attention patterns or learn novel tasks requiring new attention patterns.
Context-based fine-tuning methods, including prompting, in-context learning, soft prompting (also known as prompt tuning), and prefix-tuning, have gained popularity due to their ability to often match the performance of full fine-tuning with a fraction of the parameters. Despite their empirical successes, there is little theoretical understanding of how these techniques influence the internal computation of the model and their expressiveness limitations. We show that despite the continuous embedding space being more expressive than the discrete token space, soft-prompting and prefix-tuning are potentially less expressive than full fine-tuning, even with the same number of learnable parameters. Concretely, context-based fine-tuning cannot change the relative attention pattern over the content and can only bias the outputs of an attention layer in a fixed direction. This suggests that while techniques like prompting, in-context learning, soft prompting, and prefix-tuning can effectively elicit skills present in the pretrained model, they may not be able to learn novel tasks that require new attention patterns.