PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
For LLM inference, PARD-2 provides a more efficient draft model training objective that directly maximizes token acceptance length, enabling faster inference without quality loss.
PARD-2 introduces a dual-mode speculative decoding framework with Confidence-Adaptive Token optimization, achieving up to 6.94x lossless acceleration on Llama3.1-8B, surpassing EAGLE-3 by 1.9x and PARD by 1.3x.
Speculative decoding accelerates Large Language Models (LLMs) inference by using a lightweight draft model to propose candidate tokens that are verified in parallel by the target model. However, existing draft model training objectives are not directly aligned with the inference-time goal of maximizing consecutive token acceptance. To address this issue, we reformulate the draft model optimization objective, shifting the focus from token prediction accuracy to the overall acceptance length. In this paper, we build upon PARD to propose PARD-2, a dual-mode speculative decoding framework with Confidence-Adaptive Token (CAT) optimization. This approach adaptively reweights each token to better align with the verification process. Notably, PARD-2 enables a single draft model to support both target-dependent and target-independent modes. Experiments across diverse models and tasks demonstrate that PARD-2 achieves up to 6.94$\times$ lossless acceleration, surpassing EAGLE-3 by 1.9$\times$ and PARD by 1.3$\times$ on Llama3.1-8B. Our code is available at https://github.com/AMD-AGI/PARD.