A Survey on Prompt Tuning
It provides a comprehensive overview for researchers and practitioners in NLP, but is incremental as it synthesizes existing work without new experimental results.
This survey reviews prompt tuning, a parameter-efficient method for adapting language models by training continuous vectors while keeping the model frozen, analyzing various approaches and identifying challenges like computational efficiency and training stability.
This survey reviews prompt tuning, a parameter-efficient approach for adapting language models by prepending trainable continuous vectors while keeping the model frozen. We classify existing approaches into two categories: direct prompt learning and transfer learning. Direct prompt learning methods include: general optimization approaches, encoder-based methods, decomposition strategies, and mixture-of-experts frameworks. Transfer learning methods consist of: general transfer approaches, encoder-based methods, and decomposition strategies. For each method, we analyze method designs, innovations, insights, advantages, and disadvantages, with illustrative visualizations comparing different frameworks. We identify challenges in computational efficiency and training stability, and discuss future directions in improving training robustness and broadening application scope.