Performance Optimization and Comparative Analysis of Generative AI Models on Advanced Accelerators
For practitioners deploying generative AI models, this work offers a comparative analysis of performance across diverse hardware, but the lack of specific numbers limits its actionable impact.
This paper addresses the deployment challenges of generative AI models (LLMs, diffusion models) by systematically optimizing and comparing their performance across heterogeneous HPC systems and accelerators, introducing a novel mixed-precision post-training quantization evaluation. The study provides insights into fine-tuning strategies and performance trade-offs, though no concrete numerical results are reported.
Generative AI models, such as Large Language Models (LLMs) and diffusion models, have demonstrated impressive performance across a wide range of tasks. Despite these advances, deployment remains challenging due to substantial memory requirements, extended inference latency, significant computational demands, and high hardware costs. These issues are further complicated when evaluating models across heterogeneous platforms, where differences in numerical formats, memory bandwidths, and software stacks interact with model architecture and workload characteristics in complex ways. To address these challenges, we present a systematic study focused on performance optimization and comparative analysis of several Generative AI models across diverse downstream tasks. This work introduces a novel mixed-precision post-training quantization evaluation, examines fine-tuning strategies, and assesses performance across modern high-performance computing (HPC) systems and advanced accelerators.