Is auxiliary losses superseded?

auxiliary losses (Mixture-of-experts routing): cited as a baseline — critiqued by newer work, not yet beaten on a benchmark here. 2 paper(s) critique it, 0 beat it on benchmarks — #200 of 1370 most-superseded. Sub-problem: cluster led by MoE-LLaVA. Newer alternatives in the same sub-problem include MoE-GRPO, Input Domain Aware MoE.

Mixture-of-experts routing

auxiliary losses

Superseded baseline#200 of 1,370 most-superseded

Cited as a baseline — critiqued by newer work, not yet beaten on a benchmark here

2 papers critique it · 0 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites auxiliary losses as a baseline.

multiple studies dai2022stablemoe,wu2024gw,wang2024auxiliary demonstrate that auxiliary losses can significantly impair training stability and model performance

— Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts

“The reliance on auxiliary losses requires careful balancing between the router loss and the task loss, which introduces trade-offs”

— Unified Sparse Mixture of Experts

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.

MoE-GRPOMoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
Mar 26, 2026
Input Domain Aware MoEInput Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
Oct 18, 2025