SWIFT

“While SWIFT only allows a fixed skipping rate, Conflayers does not limit the search space to a certain number of layers to skip and expands the exploration set to any number of layers below or above a pre-defined threshold while conditioning the search on the performance of the draft model.”

“requires substantial workload-specific tuning to isolate layers and operations that can be skipped while maintaining high token acceptance rates”

“The most closely related framework to ours is SWIFT (Xia et al., 2025), which adaptively selects subsets of layers to skip during inference under a speculative decoding paradigm. By treating the same LLM as both draft and verifier via dynamic layer selection, SWIFT achieves lossless acceleration without introducing new modules or supervision. However, SWIFT still requires iterative Bayesian optimization to identify the optimal layer subsets, which can be computationally expensive prior to deployment.”

“All existing self-speculative methods share a common assumption: the model is a homogeneous stack of similar layers, and the drafting strategy consists of skipping or shortcutting some of these layers. This assumption breaks down in hybrid architectures, where layers contain fundamentally different computational components.”