Method Drift›Parameter-efficient fine-tuning (LoRA family)
FedEx-LoRA
FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation ModelsParameter-efficient fine-tuning (LoRA family) · first seen Oct 12, 2024
superseded — cited as a baseline and beaten by newer methods
2 papers critique it · 2 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites FedEx-LoRA as a baseline.
“the method substantially increases the communication cost of fine-tuning since the updated model weights also have to be communicated every round.”
— Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning“This cause extreme communication overhead, which totally eliminates the lightweight advantage of LoRA”
— Communication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank Adaptation
Beaten on benchmarks
Head-to-head results where a newer method reports beating FedEx-LoRA. Values are copied from the source paper's tables — verify against the cited paper.
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · I.I.D. [N_total = 1.2M, 20 Clients]
84.42 vs 80.56
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Non-I.I.D. [N_total = 1.2M, 20 Clients]
76.22 vs 67.45
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · I.I.D. [N_total = 1.2M, 50 Clients]
84.02 vs 77.82
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Non-I.I.D. [N_total = 1.2M, 50 Clients]
73.80 vs 66.58
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · I.I.D. [N_total = 2.4M, 20 Clients]
85.04 vs 79.38
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Non-I.I.D. [N_total = 2.4M, 20 Clients]
77.20 vs 50.47
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · I.I.D. [N_total = 2.4M, 50 Clients]
85.55 vs 79.42
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Non-I.I.D. [N_total = 2.4M, 50 Clients]
77.81 vs 57.86
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · I.I.D. [N_total = 4.7M, 20 Clients]
69.29 vs 68.59
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Non-I.I.D. [N_total = 4.7M, 20 Clients]
66.45 vs 65.11
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Average [20 clients / lower parameter budget]
83.90 vs 82.94
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
RAVAN beats FedEx-LoRA · Average [20 clients / higher parameter budget]
85.72 vs 84.38
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.
- Shift-Dependent AsymmetryShift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical SegmentationJun 7, 2026
- May 7, 2026
- Mar 3, 2026
- Feb 27, 2026
- Nov 23, 2025
- compute-efficient continual pre-training with LoRALow-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-StudyOct 26, 2025
- personalized federated fine-tuning method with orthogonal LoRA adaptersPersonalized Federated Fine-Tuning of Vision Foundation Models for HealthcareOct 14, 2025
- FLoRA-NACommunication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank AdaptationSep 30, 2025