Method Drift›Mixture-of-experts routing
ST-MoE
ST-MoE: Designing Stable and Transferable Sparse Expert ModelsMixture-of-experts routing · first seen Feb 17, 2022
superseded — cited as a baseline and beaten by newer methods
0 papers critique it · 2 beat it on benchmarks
Beaten on benchmarks
Head-to-head results where a newer method reports beating ST-MoE. Values are copied from the source paper's tables — verify against the cited paper.
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · MMVet [StableLM-1.6B + CLIP-336]
29.5 vs 29.3
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · MME-per [StableLM-1.6B + CLIP-336]
1372.0 vs 1364.5
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · PoPE [StableLM-1.6B + CLIP-336]
86.8 vs 85.4
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · SQA^I [StableLM-1.6B + CLIP-336]
65.1 vs 61.3
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · TextVQA [StableLM-1.6B + CLIP-336]
51.9 vs 50.3
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · VizWiz [StableLM-1.6B + CLIP-336]
43.1 vs 38.2
- Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
IDA-MoE beats ST-MoE · MMB [StableLM-1.6B + CLIP-336]
63.2 vs 59.5
- Hierarchical Mixture-of-Experts with Two-Stage Optimization
Hi-MoE beats ST-MoE · PPL [nanoGPT on OpenWebText]
2.947 vs 2.981
Newer alternatives
Recent methods in the same sub-problem, not yet superseded in the knowledge base.