Method Drift›Parameter-efficient fine-tuning (LoRA family)
FedIT
Parameter-efficient fine-tuning (LoRA family)
superseded — cited as a baseline and beaten by newer methods
3 papers critique it · 9 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites FedIT as a baseline.
“FedIT aggregates A and B independently: equation W = BA = (p_0 B_0 + p_1 B_1)(p_0 A_0 + p_1 A_1) which introduces additional issues for federated fine-tuning. The intermediate term obtained by the cross-product of LoRA modules from different clients introduces unexpected noise in the model aggregation.”
— FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations“The current state-of-the-art, Federated Instruction Tuning (FedIT, FedIT), uses conventional federated aggregation to average the low-rank matrices $A$ and $B$ individually. The resulting update matrix which is formed post aggregation is thus the product of the averaged matrices $A$ and $B$. However, the ideal update should be the average of the products of the low-rank adapters $A$ and $B$. The discrepancy results from the fact that "the average of the products is not equal to the product of the averages".”
— FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models“FedIT improves communication but requires homogeneous ranks”
— ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation
Beaten on benchmarks
Head-to-head results where a newer method reports beating FedIT. Values are copied from the source paper's tables — verify against the cited paper.
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · Average accuracy [3 clients, r=4, h=0.5]
0.8932 vs 0.8712
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · Average accuracy [10 clients, r=4, h=0.5]
0.8818 vs 0.8362
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · Average accuracy [50 clients, r=4, h=0.5]
0.8734 vs 0.7680
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [r=2]
0.877 vs 0.844
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [r=4]
0.876 vs 0.866
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [r=8]
0.877 vs 0.861
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [r=16]
0.865 vs 0.855
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [r=24]
0.872 vs 0.857
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [h=100 (homogeneous)]
0.884 vs 0.868
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [h=1 (heterogeneous)]
0.873 vs 0.864
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · MNLI accuracy [h=0.5 (heterogeneous)]
0.876 vs 0.866
- FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
FedRot-LoRA beats FedIT · pass@1 score [Code generation (HumanEval)]
0.4088 vs 0.2877
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.
- Shift-Dependent AsymmetryShift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical SegmentationJun 7, 2026
- May 7, 2026
- Mar 3, 2026
- Feb 27, 2026
- Nov 23, 2025
- compute-efficient continual pre-training with LoRALow-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-StudyOct 26, 2025
- personalized federated fine-tuning method with orthogonal LoRA adaptersPersonalized Federated Fine-Tuning of Vision Foundation Models for HealthcareOct 14, 2025
- FLoRA-NACommunication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank AdaptationSep 30, 2025