Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches
This work tackles the problem of enabling efficient LLM deployment at network edges for applications requiring local data processing, but it is incremental as it reviews existing methods rather than introducing new ones.
The paper addresses the challenge of fine-tuning and deploying large language models (LLMs) on edge devices with limited memory, providing a comprehensive overview of memory-efficient fine-tuning and model compression techniques to reduce operational costs.
Since the release of GPT2-1.5B in 2019, the large language models (LLMs) have evolved from specialized deep models to versatile foundation models. While demonstrating remarkable zero-shot ability, the LLMs still require fine-tuning on local datasets and substantial memory for deployment over the network edges. Traditional first-order fine-tuning techniques require significant GPU memory that exceeds the capacity of mainstream hardware. Besides, the LLMs have been expanded beyond text generation to create images, audio, video, and multi-modal content, necessitating careful investigation of efficient deployment strategies for large-scale foundation models. In response to these challenges, model fine-tuning and model-compression techniques have been developed to support the sustainable growth of LLMs by reducing both operational and capital expenditures. In this work, we provide a comprehensive overview of prevalent memory-efficient fine-tuning methods for deployment at the network edge. We also review state-of-the-art literature on model compression, offering insights into the deployment of LLMs at network edges.