OWLed: Outlier-weighed Layerwise Pruning for Efficient Autonomous Driving Framework
This addresses the problem of high computational demands for real-world autonomous driving systems, offering an incremental improvement through novel pruning techniques.
The paper tackles the computational infeasibility of deploying Large Language Models (LLMs) locally in autonomous driving by introducing OWLed, an outlier-weighted layerwise pruning framework that reduces model size without fine-tuning, achieving superior performance in perception, action prediction, and language understanding while lowering computational requirements.
The integration of Large Language Models (LLMs) into autonomous driving systems offers promising enhancements in environmental understanding and decision-making. However, the substantial computational demands of deploying LLMs locally on vehicles render this approach unfeasible for real-world automotive applications. To address this challenge, we introduce OWLed, the Outlier-Weighed Layerwise Pruning for Efficient Autonomous Driving Framework that leverages outlier-weighted layerwise sparsity for model compression. Our method assigns non-uniform sparsity ratios to different layers based on the distribution of outlier features, significantly reducing the model size without the need for fine-tuning. To ensure the compressed model adapts well to autonomous driving tasks, we incorporate driving environment data into both the calibration and pruning processes. Our empirical studies reveal that the encoder component is more sensitive to pruning than the LLM, highlighting its critical role in the system. Experimental results demonstrate that OWLed outperforms existing methods in perception, action prediction, and language understanding while substantially lowering computational requirements. These findings underscore the potential of combining advanced pruning techniques with LLMs to develop efficient and robust autonomous driving systems capable of handling complex scenarios. Code will be made publicly available.