P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
This addresses the latency issue in speculative decoding for reasoning LLMs by enabling efficient parallel drafting, though it is incremental as it builds on the EAGLE framework.
The paper tackled the problem of training parallel drafting models for speculative decoding in reasoning LLMs, which suffer from quadratic scaling complexity with sequence length and parallel positions, by introducing P-EAGLE with a learnable shared hidden state and scalable training techniques, achieving speedups of 1.10-1.36x over autoregressive EAGLE-3 across various models.
Reasoning LLMs produce longer outputs, requiring speculative decoding drafters trained on extended sequences. Parallel drafting - predicting multiple tokens per forward pass - offers latency benefits over sequential generation, but training complexity scales quadratically with the product of sequence length and parallel positions, rendering long-context training impractical. We present P(arallel)-EAGLE, which transforms EAGLE from autoregressive to parallel multi-token prediction via a learnable shared hidden state. To scale training to long contexts, we develop a framework featuring attention mask pre-computation and sequence partitioning techniques, enabling gradient accumulation within individual sequences for parallel-prediction training. We implement P-EAGLE in vLLM and demonstrate speedups of 1.10-1.36x over autoregressive EAGLE-3 across GPT-OSS 120B, 20B, and Qwen3-Coder 30B.