Comparing energy consumption and accuracy in text classification inference
It addresses energy efficiency concerns in NLP for sustainable AI development, offering insights for researchers and practitioners, but is incremental as it extends prior work on training to inference.
This study evaluated the trade-offs between accuracy and energy consumption in text classification inference across various models and hardware, finding that the best-performing model can also be energy-efficient, while larger LLMs consume significantly more energy with lower accuracy, with energy usage ranging from <mWh to >kWh.
The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sustainability. While prior research has largely focused on energy consumption during model training, the inference phase has received comparatively less attention. This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference across various model architectures and hardware configurations. Our empirical analysis shows that the best-performing model in terms of accuracy can also be energy-efficient, while larger LLMs tend to consume significantly more energy with lower classification accuracy. We observe substantial variability in inference energy consumption ($<$mWh to $>$kWh), influenced by model type, model size, and hardware specifications. Additionally, we find a strong correlation between inference energy consumption and model runtime, indicating that execution time can serve as a practical proxy for energy usage in settings where direct measurement is not feasible. These findings have implications for sustainable AI development, providing actionable insights for researchers, industry practitioners, and policymakers seeking to balance performance and resource efficiency in NLP applications.