Brevity is the soul of sustainability: Characterizing LLM response lengths
This addresses energy efficiency for LLM users, but is incremental as it builds on existing prompt-engineering techniques.
The paper tackled the problem of LLMs producing unnecessarily long responses that waste energy during inference, and found that simple prompt-engineering strategies can reduce response lengths while preserving quality, achieving 25-60% energy optimization.
A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for inference optimization, output compression remains relatively unexplored, with only a few preliminary efforts addressing this aspect. In this work, we first benchmark 12 decoder-only LLMs across 5 datasets, revealing that these models often produce responses that are substantially longer than necessary. We then conduct a comprehensive quality assessment of LLM responses, formally defining six information categories present in LLM responses. We show that LLMs often tend to include redundant or additional information besides the minimal answer. To address this issue of long responses by LLMs, we explore several simple and intuitive prompt-engineering strategies. Empirical evaluation shows that appropriate prompts targeting length reduction and controlling information content can achieve significant energy optimization between 25-60\% by reducing the response length while preserving the quality of LLM responses.