Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

Prompt Tuning

Superseded baseline#11 of 1,113 most-superseded

Superseded — cited as a baseline and beaten by newer methods

9 papers critique it · 6 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites Prompt Tuning as a baseline.

prompt-based PEFT methods, such as Prompt Tuning~lester2021power and Prefix-Tuning~li2021prefix, have been widely applied to Transformers but fail to adapt effectively to SSMs~galim2024parameter
— State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for State Space Models
“Compared to methods like LoRA, fine-tuning approaches including Prompt Tuning, P-Tuning, and IA$^{3}$, although designed with fewer parameters, struggle to effectively capture the patterns of few samples and generalize to more samples in scenarios where data is scarce”
— CoLA: Collaborative Low-Rank Adaptation
“it is acknowledged that training the prompt vectors in few-shot settings is prone to instability and exhibits slow convergence, making it challenging to generalize to large language models.”
— Derivative-Free Optimization for Low-Rank Adaptation in Large Language Models
“prompt tuning converges to a higher loss (left), and performs poorly compared to LoRA (right)”
— Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need
“P-Tuning and Prompt Tuning are bound to the self-attention mechanism”
— PEFT-MuTS: A Multivariate Parameter-Efficient Fine-Tuning Framework for Remaining Useful Life Prediction based on Cross-domain Time Series Representation Model
“However, even when learnable, adaptable prompts often struggle to capture the heterogeneity within source task distributions, possibly due to the limitations in expressive capacity imposed by their form and length”
— Efficient Knowledge Transfer in Multi-Task Learning through Task-Adaptive Low-Rank Representation
“These methods generally utilize trainable virtual tokens within an adapter, which may not have essential semantic significance and require extensive training to acquire domain-specific knowledge efficiently. Consequently, the performance of these techniques may not be optimal, particularly when dealing with complex tasks, and extensive training is necessary to achieve optimal performance.”
— Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning
“Additionally, reserving part of the sequence length for adaptation may reduce the effective input length available for the downstream task, potentially limiting performance.”
— LoRA-Mini : Adaptation Matrices Decomposition and Selective Training
“prompt tuning underperforms even the linear probing baseline in the S3DIS 6-fold evaluation, revealing the penalty of ignoring spatial structure during fine-tuning”
— On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation

Beaten on benchmarks

Head-to-head results where a newer method reports beating Prompt Tuning. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.