PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning
Provides a simple, low-cost method to improve one-shot pruning for LLMs, though gains are architecture-dependent and incremental over existing approaches.
PALS adjusts per-layer sparsity in LLM pruning based on activation percentiles, achieving 10.96 perplexity on LLaMA-2-7B at 50% sparsity versus 12.92 for uniform Wanda, with negligible overhead and no fine-tuning.
One-shot pruning methods like Wanda and SparseGPT apply the same sparsity ratio to every layer of a transformer, ignoring known variation in layer importance. We propose PALS (Percentile-Aware Layerwise Sparsity), which adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to $\pm 5\%$ around the target ratio. On LLaMA-2-7B at 50\% sparsity, PALS achieves 10.96 WikiText-2 perplexity versus 12.92 for uniform Wanda (mean over 9 runs, $p < 0.001$). The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none. We also find that gradient-based allocation -- the seemingly more principled approach -- produces results worse than random, suggesting that gradient magnitude does not predict the impact of discrete weight removal. PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.