Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

QLoRA

QLoRA: Efficient Finetuning of Quantized LLMs

Superseded baseline#9 of 1,113 most-superseded · first seen May 23, 2023

Superseded — cited as a baseline and beaten by newer methods

5 papers critique it · 14 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites QLoRA as a baseline.

Standard PEFT methods like LoRA hu2022lora and QLoRA dettmers2023qlora typically rely on uniform application across layers.
— RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models
“improves computational efficiency and reduces resource consumption through dynamic quantization and advanced strategies, though it may potentially impact model accuracy.”
— SSMLoRA: Enhancing Low-Rank Adaptation with State Space Model
“4-bit LLaMA-30B with finetuned LoRA even fails to achieve the accuracy of the original counterpart without finetuning (57.7% vs. 58.2% on MMLU)”
— Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
“Through comprehensive experiments on both summarization and classification tasks, we demonstrate that LoRAN with Sinter achieves consistent improvements over strong baselines, including QLoRA (dettmers2023qlora) and its variants.”
— Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations
“QLoRA does not maintain quantized at inference since the quantized weights need to be converted to again so as to be merged with the LoRA weights.”
— IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models

Beaten on benchmarks

Head-to-head results where a newer method reports beating QLoRA. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.