Method Drift›Parameter-efficient fine-tuning (LoRA family)
Superseded baseline#95 of 1,113 most-superseded
LISA
LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern TransformationParameter-efficient fine-tuning (LoRA family) · first seen Aug 5, 2018
superseded — cited as a baseline and beaten by newer methods
3 papers critique it · 0 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites LISA as a baseline.
“Although effective, these methods require substantial storage equivalent to the full model since all parameters are being updated. Furthermore, these approaches do not deeply explore joint use with PEFT and have employed relatively simple selection strategies, limiting their performance.”
— Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models“Layer-level methods such as LISA pan2024lisa and LoRA-drop zhou2024loradrop attempt selective adaptation, but rely on stochastic layer sampling or post-hoc pruning, both of which add overhead or require a full training pass before selection is fixed.”
— FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning“LISA pan2024lisa reduces memory usage by updating random subsets of layers, but does not model layerwise heterogeneity and modifies base model parameters, limiting compatibility with reusable adapters and large-scale serving”
— Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.