Scholar Feed

Parameter-efficient fine-tuning (LoRA family)

FedIT

Superseded baseline#16 of 1,113 most-superseded

Superseded — cited as a baseline and beaten by newer methods

3 papers critique it · 9 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites FedIT as a baseline.

FedIT aggregates A and B independently: equation W = BA = (p_0 B_0 + p_1 B_1)(p_0 A_0 + p_1 A_1) which introduces additional issues for federated fine-tuning. The intermediate term obtained by the cross-product of LoRA modules from different clients introduces unexpected noise in the model aggregation.
— FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
“The current state-of-the-art, Federated Instruction Tuning (FedIT, FedIT), uses conventional federated aggregation to average the low-rank matrices $A$ and $B$ individually. The resulting update matrix which is formed post aggregation is thus the product of the averaged matrices $A$ and $B$. However, the ideal update should be the average of the products of the low-rank adapters $A$ and $B$. The discrepancy results from the fact that "the average of the products is not equal to the product of the averages".”
— FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models
“FedIT improves communication but requires homogeneous ranks”
— ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation

Beaten on benchmarks

Head-to-head results where a newer method reports beating FedIT. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.