Deploying Open-Source Large Language Models: A performance Analysis
This work addresses deployment challenges for private and public groups using open-source LLMs, but it is incremental as it applies existing methods to new data.
The study tackled the problem of unknown deployment requirements for open-source large language models by comparing the performance of models like Mistral and LLaMa across different GPUs using vLLM, providing concrete data to help groups evaluate hardware-based performance.
Since the release of ChatGPT in November 2022, large language models (LLMs) have seen considerable success, including in the open-source community, with many open-weight models available. However, the requirements to deploy such a service are often unknown and difficult to evaluate in advance. To facilitate this process, we conducted numerous tests at the Centre Inria de l'Université de Bordeaux. In this article, we propose a comparison of the performance of several models of different sizes (mainly Mistral and LLaMa) depending on the available GPUs, using vLLM, a Python library designed to optimize the inference of these models. Our results provide valuable information for private and public groups wishing to deploy LLMs, allowing them to evaluate the performance of different models based on their available hardware. This study thus contributes to facilitating the adoption and use of these large language models in various application domains.