EAGLE-3

“The method that relies on training an auxiliary draft model demonstrate promising acceleration potential, but its inference behavior is inherently coupled to the training process, limiting adaptability across diverse reasoning scenarios.”

“State-of-the-art tree-based methods, such as eagle, construct the draft tree via a rigid layer-wise expansion mechanism.”

“Conversely, we also observe significant instability, with performance dropping to a 0.70× slowdown in the worst-case scenario (Eagle-3 of L31-8B on SWE-bench).”

“current autoregressive drafter designs introduce limitations. While approaches such as EAGLE3 significantly lower the per-step drafting cost by using a single customized layer, the drafting process is still inherently autoregressive. This sequential dependency forces the drafter to spend nearly 20\%--40\% of the total inference time, thereby fundamentally limiting the achievable acceleration, causing the drafting stage, especially the drafting forward cost, to emerge as a new bottleneck in speculative decoding.”

“However, these approaches primarily focus on optimizing draft generation while the verification phase — which constitutes 67-90% of total computation — receives limited attention for fine-grained adaptation.”

“It is plausible that EAGLE-3, trained primarily on English corpora, fails to simulate the target model's distribution accurately in Chinese reasoning tasks. Such data distribution mismatches, rooted in differences in post-training procedures or training corpora, generally constrain the robustness of model-based SD methods.”

“each added tree depth still costs one more sequential drafter round”

“Training-based method EAGLE-3 suffers significant degradation under OOD conditions.”

“existing draft models, including the EAGLE series, have a shared drawback: they generate draft tokens by conditioning solely on the current prefix. This design is prone to error accumulation.”

“we find that the high acceptance length of EAGLE3 is not viable when the context length is beyond the trained context window”

“state-of-the-art methods like EAGLE-3 still rely on autoregressive drafting. This serial drafting process is not only inherently inefficient but also susceptible to error accumulation, which effectively caps achievable speedups at approximately 2-3×”

“However, these methods are typically evaluated at 2K context with full-cache drafting~liu2026illusion, and do not address the sparse/full mismatch when a sparse KV cache constrains the drafter in longer contexts~yang2025longspec.”