Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

InfLoRA

InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning

Superseded baseline#18 of 1,113 most-superseded · first seen Mar 30, 2024

Superseded — cited as a baseline and beaten by newer methods

3 papers critique it · 9 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites InfLoRA as a baseline.

However, as the number of sequential tasks increases, the continual expansion of prompts or LoRA components, along with the growing storage costs of sample features, becomes inefficient and unsustainable.
— SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
“While they correctly identify the need for orthogonality between tasks, they attempt to enforce it at a high level without addressing a fundamental flaw in the LoRA optimization process itself. Specifically, independent Euclidean updates to the low-rank factors ($A$ and $B$) cause their composite effect to deviate from any intended orthogonal path.”
— Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
“While both approaches are effective, they differ in managing parameter updates. MoE-based approaches aggregate task-specific LoRA weights via attention mechanisms, whereas orthogonality-based methods regulate LoRA parameter updates by constraining gradient update directions. However, neither method directly examines how parameter shifts evolve across tasks (i.e., the internal dynamics of parameter space), which is a crucial yet underexplored factor in model forgetting.”
— Resolving Conflicts in Lifelong Learning via Aligning Updates in Subspaces

Beaten on benchmarks

Head-to-head results where a newer method reports beating InfLoRA. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.