Method Drift›Parameter-efficient fine-tuning (LoRA family)
QLoRA
QLoRA: Efficient Finetuning of Quantized LLMsParameter-efficient fine-tuning (LoRA family) · first seen May 23, 2023
superseded — cited as a baseline and beaten by newer methods
5 papers critique it · 14 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites QLoRA as a baseline.
“Standard PEFT methods like LoRA hu2022lora and QLoRA dettmers2023qlora typically rely on uniform application across layers.”
— RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models“improves computational efficiency and reduces resource consumption through dynamic quantization and advanced strategies, though it may potentially impact model accuracy.”
— SSMLoRA: Enhancing Low-Rank Adaptation with State Space Model“4-bit LLaMA-30B with finetuned LoRA even fails to achieve the accuracy of the original counterpart without finetuning (57.7% vs. 58.2% on MMLU)”
— Accurate LoRA-Finetuning Quantization of LLMs via Information Retention“Through comprehensive experiments on both summarization and classification tasks, we demonstrate that LoRAN with Sinter achieves consistent improvements over strong baselines, including QLoRA (dettmers2023qlora) and its variants.”
— Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations“QLoRA does not maintain quantized at inference since the quantized weights need to be converted to again so as to be merged with the LoRA weights.”
— IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models
Beaten on benchmarks
Head-to-head results where a newer method reports beating QLoRA. Values are copied from the source paper's tables — verify against the cited paper.
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · MMLU accuracy [LLaMA-7b]
39.7 vs 38.8
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · MMLU accuracy [Falcon-40b]
57.1 vs 55.2
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · BLEU score [Falcon-40b, Web-GLM, Rank 2]
56.0 vs 19.9
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · Exact match [Falcon-40b, GSM8k, Rank 8]
30.6 vs 15.1
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · Exact match [GSM8K, LLaMA-7b, Rank 8]
16.76 vs 0.0
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · Exact match [TriviaQA, LLaMA-7b, Rank 32]
21.65 vs 0.0
- QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
QDyLoRA beats QLoRA · Exact match [GSM8K, LLaMA2-13b, Rank 8]
25.55 vs 0.0
- FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
QFuRA beats QLoRA · Avg [LLaMA-3-8B]
87.30 vs 83.89
- FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
QFuRA beats QLoRA · GSM8K [LLaMA-3-70B]
83.78 vs 81.27
- MSPLoRA: A Multi-Scale Pyramid Low-Rank Adaptation for Efficient Model Fine-Tuning
MSPLoRA beats QLoRA · Avg. [Avg]
32.09 vs 30.61
- Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
QCorDA beats QLoRA · GSM8k [Quantized, LLaMA-2-7B, rank 128]
51.86 vs 38.89
- Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
QCorDA beats QLoRA · Math [Quantized, LLaMA-2-7B, rank 128]
7.68 vs 4.7
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.
- May 29, 2026
- May 28, 2026
- May 19, 2026
- May 15, 2026
- May 12, 2026
- May 11, 2026
- May 11, 2026
- May 8, 2026
- May 5, 2026
- May 5, 2026
- May 5, 2026
- RDP LoRARDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language ModelsApr 21, 2026