Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
This addresses the problem of high energy consumption in data centers for AI practitioners and providers, but it is incremental as it focuses on optimizing existing knobs rather than introducing new methods.
The paper tackles the challenge of energy efficiency in large language model (LLM) inference serving by analyzing trade-offs between energy usage and performance under service-level agreements, offering insights to optimize energy without compromising latency or throughput.
With the ubiquitous use of modern large language models (LLMs) across industries, the inference serving for these models is ever expanding. Given the high compute and memory requirements of modern LLMs, more and more top-of-the-line GPUs are being deployed to serve these models. Energy availability has come to the forefront as the biggest challenge for data center expansion to serve these models. In this paper, we present the trade-offs brought up by making energy efficiency the primary goal of LLM serving under performance SLOs. We show that depending on the inputs, the model, and the service-level agreements, there are several knobs available to the LLM inference provider to use for being energy efficient. We characterize the impact of these knobs on the latency, throughput, as well as the energy. By exploring these trade-offs, we offer valuable insights into optimizing energy usage without compromising on performance, thereby paving the way for sustainable and cost-effective LLM deployment in data center environments.