Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards
This work addresses the need for lightweight prompt optimization in NLP, offering an incremental improvement over existing derivative-free methods.
The paper tackles the problem of derivative-free prompt learning by proposing Clip-Tuning, which uses frozen thinned networks from pre-trained language models to obtain a mixture of rewards, achieving parity with gradient-based methods on seven benchmarks under few-shot settings.
Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did not take full advantage of the over-parameterized characteristics of large pre-trained language models (PLMs). In this paper, we propose Clip-Tuning, a simple yet effective method that adopts diverse frozen "thinned" networks of PLMs to obtain a mixture of rewards and thus advance the derivative-free prompt learning. The thinned networks consist of all the hidden units that survive a stationary dropout strategy, whose inference predictions reflect an ensemble of partial views over prompted training samples. Our method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings.