UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods
For practitioners deploying large MoE models on production superpods, UBEP improves communication efficiency and inference speed.
UBEP addresses three bottlenecks in MoE model communication on high-bandwidth superpods (e.g., NVL72, CloudMatrix384): execution serialization, synchronization overhead, and load imbalance. It reduces All-to-All latency by up to 52.4% and MoE inference TPOT by up to 11.1%.
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.