Is MixLoRA superseded?

MixLoRA (Mixture-of-experts routing): superseded — cited as a baseline and beaten by newer methods. 0 paper(s) critique it, 3 beat it on benchmarks — #28 of 1370 most-superseded. Sub-problem: cluster led by HydraLoRA. Newer alternatives in the same sub-problem include PARAMΔ Integration into Upcycled MoE, MEMIT-like framework for MoE, HELLoRA, SDG-MoE, Marco-MoE.

Mixture-of-experts routing

MixLoRA

MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts

Superseded baseline#28 of 1,370 most-superseded · first seen Apr 22, 2024

Superseded — cited as a baseline and beaten by newer methods

0 papers critique it · 3 beat it on benchmarks

Beaten on benchmarks

Head-to-head results where a newer method reports beating MixLoRA. Values are copied from the source paper's tables — verify against the cited paper.

GMoE beats MixLoRA
0.48 vs 1.04
Stability Average (Std) · [Qwen2]
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
MoORE (L=8) beats MixLoRA
85.11 vs 82.55
Overall · [CSR-MTL multi-task adaptation]
MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
GraphMoE(MixLoRA) beats MixLoRA
84.9 vs 82.6
AVG · [LoRA+MoE baseline methods]
GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.