LayerSkip

“However, self-speculative decoding, which uses the same architecture for both draft and target models, inherently limits speedup.”

“The performance of EESD hinges on several factors: the early-exit position (which affects draft speed), the draft accuracy (i.e., token acceptance rate), and the number of drafted tokens per step (draft length). Notably, a trade-off exists that more layers involved in drafting improve the acceptance rate but also increase computational cost.”

“Prior works have relied on static configuration of E and γ, selected via offline grid search. This introduces two key limitations. First, the optimal E and γ vary significantly across tasks; configurations tuned for one task often underperform on others.”

“However, applying SSD directly to multimodal models is challenging, as deeper layers are often essential for capturing cross-modal interactions. Simply skipping layers and forwarding shallow outputs to the LM Head degrades performance.”

“More recent self-contained designs like Self-Speculative Decoding (Self-SD) (Zhang et al., 2024) and LayerSkip (Elhoushi et al., 2024) further attempt to reduce computational redundancy by skipping non-critical layers during inference. While these methods highlight the potential of exploiting structural redundancy within LLMs, they typically rely on offline optimization or fine-tuning to identify task-dependent layer configurations, making them less practical in real-world deployment.”

“While self-speculation simplifies the deployment pipeline, it often provides limited acceleration.”

“All existing self-speculative methods share a common assumption: the model is a homogeneous stack of similar layers, and the drafting strategy consists of skipping or shortcutting some of these layers. This assumption breaks down in hybrid architectures, where layers contain fundamentally different computational components.”