MoLA (Mixture-of-experts routing): superseded — cited as a baseline and beaten by newer methods. 1 paper(s) critique it, 3 beat it on benchmarks — #30 of 1370 most-superseded. Sub-problem: cluster led by HydraLoRA. Newer alternatives in the same sub-problem include PARAMΔ Integration into Upcycled MoE, MEMIT-like framework for MoE, HELLoRA, SDG-MoE, Marco-MoE.

Superseded baseline#30 of 1,370 most-superseded

MoLA

Mixture-of-experts routing

superseded — cited as a baseline and beaten by newer methods

1 papers critique it · 3 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites MoLA as a baseline.

“While effective, this approach introduces three inefficiencies: (i) parameter explosion—with E experts, methods like MoLA or LoRAMoE replicate adapters, causing parameters to grow with E.”
— LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning

Beaten on benchmarks

Head-to-head results where a newer method reports beating MoLA. Values are copied from the source paper's tables — verify against the cited paper.

DeltaMoE beats MoLA · Avg. [Expanded Languages (hu, sr, bn)]
74.97 vs 73.21
A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE
DeltaMoE beats MoLA · Avg. [Original Languages (en, zh, es, fr)]
81.17 vs 79.62
A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE
GraphMoE(MixLoRA) beats MoLA · AVG [LoRA+MoE baseline methods]
84.9 vs 82.9
GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism
GraphMoE(MoLA) beats MoLA · AVG [MoLA variant]
85.0 vs 82.9
GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism
GMoE beats MoLA · Accuracy Average [Llama3]
80.52 vs 78.24
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
GMoE beats MoLA · Stability Average (Std) [Llama3]
0.45 vs 0.79
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
GMoE beats MoLA · Accuracy Average [Qwen2]
83.29 vs 82.05
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
GMoE beats MoLA · Stability Average (Std) [Qwen2]
0.48 vs 1.18
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
GMoE beats MoLA · Accuracy Average [Yi-1.5]
83.28 vs 82.01
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration
GMoE beats MoLA · Stability Average (Std) [Yi-1.5]
0.38 vs 0.54
GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration

Newer alternatives

Recent methods in the same sub-problem, not yet superseded in the knowledge base.