Parameter-efficient fine-tuning (LoRA family)

SWAG-LoRA

Superseded baseline#239 of 1,113 most-superseded

Cited as a baseline — critiqued by newer work, not yet beaten on a benchmark here

2 papers critique it · 0 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites SWAG-LoRA as a baseline.

the SWAG-LoRA method~onal2024gaussian needs additional computation to obtain an estimation of the posterior.
Variational Low-Rank Adaptation Using IVON
One major issue is the higher storage and memory requirements, as Bayesian methods often require additional parameters to model uncertainty, particularly those involving covariance estimation, such as SWAG-LoRA.
Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRA