Speed and Conversational Large Language Models: Not All Is About Tokens per Second

This research addresses the need for more nuanced speed benchmarks in LLMs for developers and researchers, though it is incremental as it focuses on comparative analysis without introducing new methods.

The study analyzed the speed of popular open-weights large language models on GPUs, finding that performance varies significantly depending on the specific task, not just token generation rates.