Revisiting Pre-processing Group Fairness: A Modular Benchmarking Framework
This work addresses a gap in fairness research for practitioners and researchers by providing a tool to evaluate data-level bias mitigation, though it is incremental as it builds on existing platforms like AIF360.
The authors tackled the lack of standardized evaluation tools for pre-processing fairness methods in machine learning by introducing FairPrep, a modular benchmarking framework that enables reproducible assessments of fairness and utility on tabular datasets.
As machine learning systems become increasingly integrated into high-stakes decision-making processes, ensuring fairness in algorithmic outcomes has become a critical concern. Methods to mitigate bias typically fall into three categories: pre-processing, in-processing, and post-processing. While significant attention has been devoted to the latter two, pre-processing methods, which operate at the data level and offer advantages such as model-agnosticism and improved privacy compliance, have received comparatively less focus and lack standardised evaluation tools. In this work, we introduce FairPrep, an extensible and modular benchmarking framework designed to evaluate fairness-aware pre-processing techniques on tabular datasets. Built on the AIF360 platform, FairPrep allows seamless integration of datasets, fairness interventions, and predictive models. It features a batch-processing interface that enables efficient experimentation and automatic reporting of fairness and utility metrics. By offering standardised pipelines and supporting reproducible evaluations, FairPrep fills a critical gap in the fairness benchmarking landscape and provides a practical foundation for advancing data-level fairness research.