Speculative decoding

Most-superseded baselines

Ranked by how many distinct papers critique or beat each method. These are the standard baselines newer work routinely measures against.

1. 1

   EAGLE-3· EAGLE-3

   EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test

   13 papers critique it · 28 beat it on benchmarks

2. 2

   EAGLE-2· EAGLE-2

   EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees

   10 papers critique it · 21 beat it on benchmarks

3. 3

   EAGLE· EAGLE-2

   EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

   19 papers critique it · 12 beat it on benchmarks

4. 4

   Medusa· EAGLE-2

   Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

   19 papers critique it · 9 beat it on benchmarks

5. 5

   Lookahead· Lookahead

   Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy

   8 papers critique it · 13 beat it on benchmarks

6. 6

   PLD· Lookahead

   PLD+: Accelerating LLM inference by leveraging Language Model Artifacts

   7 papers critique it · 13 beat it on benchmarks

7. 7

   REST· Lookahead

   REST: Retrieval-Based Speculative Decoding

   6 papers critique it · 9 beat it on benchmarks

8. 8

   SpecInfer· SpecInfer

   SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification

   10 papers critique it · 4 beat it on benchmarks

9. 9

   Speculative Sampling· Lookahead

   Speculative Sampling for Parametric Temporal Point Processes

   2 papers critique it · 10 beat it on benchmarks

10. 10

    DFlash· EAGLE-3

    DFlash: Block Diffusion for Flash Speculative Decoding

    4 papers critique it · 6 beat it on benchmarks

11. 11

    LayerSkip· SpecInfer

    LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding

    7 papers critique it · 3 beat it on benchmarks

12. 12

    FR-Spec· FR-Spec

    FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling

    4 papers critique it · 5 beat it on benchmarks