Watt For What: Rethinking Deep Learning's Energy-Performance Relationship
This addresses the environmental and equity issues in deep learning by enabling more sustainable and accessible practices, though it is incremental in proposing a new metric rather than a fundamental breakthrough.
The paper tackles the problem of deep learning's high energy consumption by proposing a metric that penalizes electricity use and analyzing accuracy-efficiency trade-offs across models and GPUs, showing that smaller, energy-efficient models can significantly expedite research while reducing environmental impact.
Deep learning models have revolutionized various fields, from image recognition to natural language processing, by achieving unprecedented levels of accuracy. However, their increasing energy consumption has raised concerns about their environmental impact, disadvantaging smaller entities in research and exacerbating global energy consumption. In this paper, we explore the trade-off between model accuracy and electricity consumption, proposing a metric that penalizes large consumption of electricity. We conduct a comprehensive study on the electricity consumption of various deep learning models across different GPUs, presenting a detailed analysis of their accuracy-efficiency trade-offs. By evaluating accuracy per unit of electricity consumed, we demonstrate how smaller, more energy-efficient models can significantly expedite research while mitigating environmental concerns. Our results highlight the potential for a more sustainable approach to deep learning, emphasizing the importance of optimizing models for efficiency. This research also contributes to a more equitable research landscape, where smaller entities can compete effectively with larger counterparts. This advocates for the adoption of efficient deep learning practices to reduce electricity consumption, safeguarding the environment for future generations whilst also helping ensure a fairer competitive landscape.