Integrating Fairness and Model Pruning Through Bi-level Optimization
This addresses the issue of fairness in model compression for applications where biased predictions are critical, though it is incremental as it builds on existing pruning techniques.
The paper tackles the problem that traditional model pruning methods can increase algorithmic biases, by proposing a fair model pruning framework that jointly optimizes pruning and fairness constraints. The result is a method that maintains performance while ensuring fairness, validated as superior to mainstream pruning strategies in experiments.
Deep neural networks have achieved exceptional results across a range of applications. As the demand for efficient and sparse deep learning models escalates, the significance of model compression, particularly pruning, is increasingly recognized. Traditional pruning methods, however, can unintentionally intensify algorithmic biases, leading to unequal prediction outcomes in critical applications and raising concerns about the dilemma of pruning practices and social justice. To tackle this challenge, we introduce a novel concept of fair model pruning, which involves developing a sparse model that adheres to fairness criteria. In particular, we propose a framework to jointly optimize the pruning mask and weight update processes with fairness constraints. This framework is engineered to compress models that maintain performance while ensuring fairness in a unified process. To this end, we formulate the fair pruning problem as a novel constrained bi-level optimization task and derive efficient and effective solving strategies. We design experiments across various datasets and scenarios to validate our proposed method. Our empirical analysis contrasts our framework with several mainstream pruning strategies, emphasizing our method's superiority in maintaining model fairness, performance, and efficiency.