Efficient Large Language Models with Zero-Shot Adjustable Acceleration
This addresses efficiency issues for real-world LLM applications, offering a novel solution for dynamic acceleration.
The paper tackles the challenge of balancing computational efficiency and performance in Large Language Models by introducing Zero-Shot Adjustable Acceleration, a method that dynamically adjusts hardware utilization during inference without fine-tuning, achieving up to 11x speedup compared to the baseline.
Using Large Language Models (LLMs) in real-world applications presents significant challenges, particularly in balancing computational efficiency with model performance. Optimizing acceleration after fine-tuning and during inference is critical for building efficient architectures. This paper introduces Zero-Shot Adjustable Acceleration, a novel training and inference method that dynamically adjusts hardware utilization during inference without requiring additional fine-tuning. The proposed approach is applied to recent LLMs and evaluated across multiple classification and text generation tasks. Experimental results demonstrate that the method supports a wide range of zero-shot acceleration and achieves up to 11x speedup compared to the baseline.