IG-Pruning: Input-Guided Block Pruning for Large Language Models
This work addresses efficient inference for LLMs in resource-constrained deployment scenarios, representing an incremental improvement over existing static pruning methods.
The paper tackles the problem of reducing computational costs in large language models (LLMs) by proposing IG-Pruning, an input-aware block-wise pruning method that dynamically selects layer masks at inference time, achieving consistent outperformance over state-of-the-art static depth pruning methods.
With the growing computational demands of large language models (LLMs), efficient inference has become increasingly critical for practical deployment. Depth pruning has emerged as a promising approach for reducing the computational costs of large language models by removing transformer layers. However, existing methods typically rely on fixed block masks, which can lead to suboptimal performance across different tasks and inputs. In this paper, we propose IG-Pruning, a novel input-aware block-wise pruning method that dynamically selects layer masks at inference time. Our approach consists of two stages: (1) Discovering diverse mask candidates through semantic clustering and L0 optimization, and (2) Implementing efficient dynamic pruning without the need for extensive training. Experimental results demonstrate that our method consistently outperforms state-of-the-art static depth pruning methods, making it particularly suitable for resource-constrained deployment scenarios.