Assessing Tenstorrent's RISC-V MatMul Acceleration Capabilities
It addresses the need for energy-efficient hardware for generative AI services, but is incremental as it benchmarks an existing accelerator against others.
This paper evaluated the Tenstorrent Grayskull e75 RISC-V accelerator for matrix multiplication in LLMs, finding it achieves a peak efficiency of 1.55 TFLOPs/Watt with BF16, offering a competitive trade-off in power and throughput compared to NVIDIA GPUs.
The increasing demand for generative AI as Large Language Models (LLMs) services has driven the need for specialized hardware architectures that optimize computational efficiency and energy consumption. This paper evaluates the performance of the Tenstorrent Grayskull e75 RISC-V accelerator for basic linear algebra kernels at reduced numerical precision, a fundamental operation in LLM computations. We present a detailed characterization of Grayskull's execution model, gridsize, matrix dimensions, data formats, and numerical precision impact computational efficiency. Furthermore, we compare Grayskull's performance against state-of-the-art architectures with tensor acceleration, including Intel Sapphire Rapids processors and two NVIDIA GPUs (V100 and A100). Whilst NVIDIA GPUs dominate raw performance, Grayskull demonstrates a competitive trade-off between power consumption and computational throughput, reaching a peak of 1.55 TFLOPs/Watt with BF16.