Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation
This addresses the barrier of computational requirements for secure, local deployment of large language models in biomedical settings, though it is incremental as it applies existing quantization techniques to new data.
The study tackled the problem of deploying large language models in healthcare by evaluating quantization's impact on 12 models across eight biomedical NLP tasks, showing it reduces GPU memory by up to 75% while preserving performance, enabling 70B-parameter models on 40GB GPUs.
Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation.