Method Drift›Parameter-efficient fine-tuning (LoRA family)
BLoB
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language ModelsParameter-efficient fine-tuning (LoRA family) · first seen Jun 17, 2024
superseded — cited as a baseline and beaten by newer methods
3 papers critique it · 2 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites BLoB as a baseline.
“it comes at the cost of needing 40% more parameters than LoRA. This can be a major memory bottleneck in high-stakes, resource-constrained deployments”
— Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference“The more recent Bayesian Low-rank adaptation by Backpropagation (BLoB) approach performs variational inference over the Low-Rank Adaptation (LoRA) parameters but requires Monte Carlo sampling to compute the likelihood term in the Evidence Lower Bound (ELBO), resulting in excessive memory consumption and computational overhead”
— Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization“BLoB wang2024blob trains Bayesian LoRA weights via backpropagation but requires mean-field assumptions over the full LoRA parameter space.”
— Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
Beaten on benchmarks
Head-to-head results where a newer method reports beating BLoB. Values are copied from the source paper's tables — verify against the cited paper.
- Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
ScalaBL beats BLoB · ECE [ARC-Challenge dataset]
9.79 vs 14.05
- Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
ScalaBL beats BLoB · ECE [ARC-Easy dataset]
7.89 vs 9.65
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
ALC(9,2) beats BLoB · ACC [ARC-Challenge]
0.908 vs 0.907
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
ALC(5,2) beats BLoB · ACC [ARC-Easy]
0.971 vs 0.965
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
ALC(5,2) beats BLoB · ACC [OBQA]
0.924 vs 0.917
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
ALC(2,1) beats BLoB · ACC [BoolQ]
0.900 vs 0.891
- On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
ALC(9,2) beats BLoB · ECE [ARC-Easy]
0.012 vs 0.015
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.