Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

BLoB

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models

Superseded baseline#44 of 1,113 most-superseded · first seen Jun 17, 2024

Superseded — cited as a baseline and beaten by newer methods

3 papers critique it · 2 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites BLoB as a baseline.

it comes at the cost of needing 40% more parameters than LoRA. This can be a major memory bottleneck in high-stakes, resource-constrained deployments
— Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
“The more recent Bayesian Low-rank adaptation by Backpropagation (BLoB) approach performs variational inference over the Low-Rank Adaptation (LoRA) parameters but requires Monte Carlo sampling to compute the likelihood term in the Evidence Lower Bound (ELBO), resulting in excessive memory consumption and computational overhead”
— Fine-tuning LLMs with variational Bayesian last layer for high-dimensional Bayesian optimization
“BLoB wang2024blob trains Bayesian LoRA weights via backpropagation but requires mean-field assumptions over the full LoRA parameter space.”
— Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

Beaten on benchmarks

Head-to-head results where a newer method reports beating BLoB. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.