Method Drift›Parameter-efficient fine-tuning (LoRA family)
HetLoRA
Parameter-efficient fine-tuning (LoRA family)
superseded — cited as a baseline and beaten by newer methods
1 papers critique it · 4 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites HetLoRA as a baseline.
“HetLoRA cho-etal-2024-heterogeneous and FlexLoRA bai2024federated allow clients to train varying-rank LoRA parameters, but the methods struggle in the presence of data heterogeneity and do not ensure exact aggregation.”
— Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
Beaten on benchmarks
Head-to-head results where a newer method reports beating HetLoRA. Values are copied from the source paper's tables — verify against the cited paper.
- HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
HSplitLoRA beats HetLoRA · BLEU [LLaMA-2-7B (homo)]
68.1 vs 66.2
- HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
HSplitLoRA beats HetLoRA · BLEU [LLaMA-2-7B (hetero)]
68.0 vs 61.3
- HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
HSplitLoRA beats HetLoRA · BLEU [GPT-2-L (homo)]
69.7 vs 68.0
- HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
HSplitLoRA beats HetLoRA · BLEU [GPT-2-L (hetero)]
69.5 vs 62.2
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · Average [Normal rank distribution]
88.34 vs 87.56
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · Average [Uniform rank distribution]
87.05 vs 86.57
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · Average [Heavy-tail rank distribution]
88.30 vs 87.92
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · ROUGE-L [Normal rank distribution, in-domain clients]
52.26 vs 50.31
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · ROUGE-L [Uniform rank distribution, in-domain clients]
49.61 vs 48.59
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · ROUGE-L [Heavy-tail rank distribution, in-domain clients]
50.42 vs 49.49
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · ROUGE-L [Normal rank distribution, unseen tasks]
37.97 vs 35.95
- Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA
GLoRA beats HetLoRA · ROUGE-L [Uniform rank distribution, unseen tasks]
34.95 vs 33.04
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.
- Shift-Dependent AsymmetryShift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical SegmentationJun 7, 2026
- May 7, 2026
- Mar 3, 2026
- Feb 27, 2026
- Nov 23, 2025
- compute-efficient continual pre-training with LoRALow-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-StudyOct 26, 2025
- personalized federated fine-tuning method with orthogonal LoRA adaptersPersonalized Federated Fine-Tuning of Vision Foundation Models for HealthcareOct 14, 2025
- FLoRA-NACommunication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank AdaptationSep 30, 2025