Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
For practitioners deploying large multimodal LLMs with Hybrid SWA and MoE, this work provides a comprehensive engineering solution to realize theoretical efficiency gains in production.
The paper presents a full-pipeline inference optimization for MiMo-V2.5 models combining Hybrid SWA, sparse MoE, and multimodal encoders, achieving strict O(W) SWA storage and high cache hit rates through KVCache optimizations, GCache infrastructure, and multimodal optimizations. It is the first large-scale production system for this composite architecture.
We present a full-pipeline inference optimization for the MiMo-V2.5 model family, which combines Hybrid Sliding Window Attention (Hybrid SWA), sparse Mixture-of-Experts (MoE), and multimodal encoders. While Hybrid SWA can ideally reduce both attention compute and KVCache storage significantly compared to Full Attention, realizing these gains in production requires substantial engineering effort. We systematically optimize the KVCache system with layerwise prefetch, SWA-aware prefix cache trees, and specialized placement strategies, achieving strict $O(W)$ SWA storage and high cache hit rates. We further build GCache, a high-performance distributed cache infrastructure with RDMA-optimized networking, and develop a KVCache-affinity router to reduce computation while preserving load balancing. We also optimize for multimodal inputs, including GPU image preprocessing, parallel video decoding, and multimodal cache sharing. Together, these optimizations constitute the first large-scale LLM serving system in production that efficiently covers the Hybrid SWA + MoE + multimodal composite architecture.