AI Application Benchmarking: Power-Aware Performance Analysis for Vision and Language Models
This work addresses energy efficiency for AI infrastructure deployment, but it is incremental as it applies existing benchmarking methods to new data and scenarios.
The authors tackled the problem of high power and energy demands in AI workloads by introducing a benchmarking framework to analyze performance-energy trade-offs for vision and language models under power capping on modern GPUs, finding that no universal optimal power cap exists as efficiency peaks vary by application type and GPU architecture.
Artificial Intelligence (AI) workloads drive a rapid expansion of high-performance computing (HPC) infrastructures and increase their power and energy demands towards a critical level. AI benchmarks representing state-of-the art workloads and their understanding in the context of performance-energy trade-offs are critical to deploy efficient infrastructures and can guide energy efficiency measures, such as power capping. We introduce a benchmarking framework with popular deep learning applications from computer vision (image classification and generation) and large language models (continued pre-training and inference) implementing modern methods. Our performance analysis focuses on throughput rather than time to "completion", which is the standard metric in HPC. We analyse performance and energy efficiency under various power capping scenarios on NVIDIA H100, NVIDIA H200, and AMD MI300X GPUs. Our results reveal that no universal optimal power cap exists, as the efficiency peak varies across application types and GPU architectures. Interestingly, the two NVIDIA GPUs which mainly differ in their HBM configuration show qualitatively different performance-energy trade-offs. The developed benchmarking framework will be released as a public tool.