Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameters
This work addresses the high computational demands and energy inefficiency of LLM inference, which is a critical issue for deploying sustainable AI systems, by introducing a novel cross-disciplinary method that bridges software engineering and machine learning.
The paper tackles the problem of optimizing LLM inference configurations for energy efficiency and sustainability by applying variability modeling techniques to systematically analyze hyperparameters, showing that this approach effectively manages complexity, reveals trade-offs, and enables accurate prediction of inference behavior from limited measurements.
Large Language Models (LLMs) are being increasingly used across a wide range of tasks. However, their substantial computational demands raise concerns about the energy efficiency and sustainability of both training and inference. Inference, in particular, dominates total compute usage, making its optimization crucial. Recent research has explored optimization techniques and analyzed how configuration choices influence energy consumption. Yet, the vast configuration space of inference servers makes exhaustive empirical evaluation infeasible due to combinatorial explosion. In this paper, we introduce a new perspective on this problem by treating LLMs as configurable systems and applying variability management techniques to systematically analyze inference-time configuration choices. We evaluate our approach on the Hugging Face Transformers library by representing generation hyperparameters and their constraints using a feature-based variability model, sampling representative configurations, measuring their energy consumption, latency, accuracy, and learning predictive models from the collected data. Our results show that variability modeling effectively manages the complexity of LLM inference configurations. It enables systematic analysis of hyperparameters effects and interactions, reveals trade-offs, and supports accurate prediction of inference behavior from a limited number of measurements. Overall, this work opens a new research direction that bridges software engineering and machine learning by leveraging variability modeling for the efficient and sustainable configuration of LLMs.