Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
It addresses the deployment challenge for practitioners in mobile and edge computing, but is incremental as it synthesizes existing methods without new results.
This survey tackles the problem of deploying large language models (LLMs) on resource-constrained devices by reviewing model compression techniques, including knowledge distillation, quantization, and pruning, to enable efficient inference.
Large Language Models (LLMs) have revolutionized many areas of artificial intelligence (AI), but their substantial resource requirements limit their deployment on mobile and edge devices. This survey paper provides a comprehensive overview of techniques for compressing LLMs to enable efficient inference in resource-constrained environments. We examine three primary approaches: Knowledge Distillation, Model Quantization, and Model Pruning. For each technique, we discuss the underlying principles, present different variants, and provide examples of successful applications. We also briefly discuss complementary techniques such as mixture-of-experts and early-exit strategies. Finally, we highlight promising future directions, aiming to provide a valuable resource for both researchers and practitioners seeking to optimize LLMs for edge deployment.