Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions
This addresses fairness concerns in compressed LLMs for opinion summarization, which could influence public views, though it is incremental as it builds on existing pruning techniques.
The study investigated how pruning large language models affects fairness in opinion summarization, finding that pruning methods impact fairness more than calibration sets, and proposed HGLA pruning, which maintained or improved fairness compared to existing methods, with human evaluations confirming its superiority.
Model compression through post-training pruning offers a way to reduce model size and computational requirements without significantly impacting model performance. However, the effect of pruning on the fairness of LLM-generated summaries remains unexplored, particularly for opinion summarisation where biased outputs could influence public views.In this paper, we present a comprehensive empirical analysis of opinion summarisation, examining three state-of-the-art pruning methods and various calibration sets across three open-source LLMs using four fairness metrics. Our systematic analysis reveals that pruning methods have a greater impact on fairness than calibration sets. Building on these insights, we propose High Gradient Low Activation (HGLA) pruning, which identifies and removes parameters that are redundant for input processing but influential in output generation. Our experiments demonstrate that HGLA can better maintain or even improve fairness compared to existing methods, showing promise across models and tasks where traditional methods have limitations. Our human evaluation shows HGLA-generated outputs are fairer than existing state-of-the-art pruning methods. Code is available at: https://github.com/amberhuang01/HGLA.