Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

LISA

LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation

Superseded baseline#95 of 1,113 most-superseded · first seen Aug 5, 2018

Cited as a baseline — critiqued by newer work, not yet beaten on a benchmark here

3 papers critique it · 0 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites LISA as a baseline.

Although effective, these methods require substantial storage equivalent to the full model since all parameters are being updated. Furthermore, these approaches do not deeply explore joint use with PEFT and have employed relatively simple selection strategies, limiting their performance.
— Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
“Layer-level methods such as LISA pan2024lisa and LoRA-drop zhou2024loradrop attempt selective adaptation, but rely on stochastic layer sampling or post-hoc pruning, both of which add overhead or require a full training pass before selection is fixed.”
— FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning
“LISA pan2024lisa reduces memory usage by updating random subsets of layers, but does not model layerwise heterogeneity and modifies base model parameters, limiting compatibility with reusable adapters and large-scale serving”
— Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.