Experience Deploying Containerized GenAI Services at an HPC Center
This work addresses the problem of enabling GenAI workloads in HPC environments for researchers and practitioners, but it is incremental as it builds on existing container and cloud technologies.
The paper tackles the challenge of deploying containerized GenAI services at an HPC center by developing a converged computing architecture that integrates HPC and Kubernetes platforms, as demonstrated through a case study deploying the Llama LLM using vLLM across multiple container runtimes.
Generative Artificial Intelligence (GenAI) applications are built from specialized components -- inference servers, object storage, vector and graph databases, and user interfaces -- interconnected via web-based APIs. While these components are often containerized and deployed in cloud environments, such capabilities are still emerging at High-Performance Computing (HPC) centers. In this paper, we share our experience deploying GenAI workloads within an established HPC center, discussing the integration of HPC and cloud computing environments. We describe our converged computing architecture that integrates HPC and Kubernetes platforms running containerized GenAI workloads, helping with reproducibility. A case study illustrates the deployment of the Llama Large Language Model (LLM) using a containerized inference server (vLLM) across both Kubernetes and HPC platforms using multiple container runtimes. Our experience highlights practical considerations and opportunities for the HPC container community, guiding future research and tool development.