Frontier: Simulating the Next Generation of LLM Inference Systems
This addresses the need for better simulation tools in AI systems research, particularly for developers and researchers working on scalable LLM inference, though it is incremental as it builds on existing simulation concepts.
The paper tackles the problem of simulating complex LLM inference systems, such as Mixture-of-Experts and disaggregated architectures, by introducing Frontier, a high-fidelity simulator that provides a unified framework for modeling these emerging paradigms and enabling design optimization.
Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like prefill/decode (PD) or attention/FFN (AF) for heterogeneous scaling. Existing simulators, architected for co-located, dense models, are unable to capture the intricate system dynamics of these emerging paradigms. We present Frontier, a high-fidelity simulator designed from the ground up for this new landscape. Frontier introduces a unified framework to model both co-located and disaggregated systems, providing native support for MoE inference with expert parallelism (EP). It enables the simulation of complex workflows like cross-cluster expert routing and advanced pipelining strategies for latency hiding. To ensure fidelity and usability, Frontier incorporates refined operator models for improved accuracy. Frontier empowers the community to design and optimize the future of LLM inference at scale.