An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
This addresses the problem of computational budget limitations for users adapting large language models to downstream tasks, though it appears incremental as it builds on existing sparse fine-tuning and pruning approaches.
The paper tackles the problem of making fine-tuning of foundation models more computationally efficient by developing a new sparse fine-tuning framework based on neural network pruning, which improves memory efficiency by 20-50% while matching the accuracy of state-of-the-art methods like LoRA variants.
Fine-tuning is an important step in adapting foundation models such as large language models to downstream tasks. To make this step more accessible to users with limited computational budgets, it is crucial to develop fine-tuning methods that are memory and computationally efficient. Sparse Fine-tuning (SpFT) and Low-rank adaptation (LoRA) are two frameworks that have emerged for addressing this problem and have been adopted widely in practice. In this work, we develop a new SpFT framework, based on ideas from neural network pruning. At a high level, we first identify ``important'' neurons/nodes using feature importance metrics from network pruning (specifically, we use the structural pruning method), and then perform fine-tuning by restricting to weights involving these neurons. Experiments on common language tasks show our method improves SpFT's memory efficiency by 20-50\% while matching the accuracy of state-of-the-art methods like LoRA's variants.