BiMi Sheets: Infosheets for bias mitigation methods
This addresses the problem of limited uptake and benchmarking difficulties for researchers and practitioners in fairness-focused ML, though it is incremental as it builds on existing documentation practices.
The paper tackles the challenge of comparing and adopting bias mitigation methods in machine learning by proposing BiMi Sheets, a uniform documentation framework, and provides a platform at bimisheet.com to facilitate their use.
Over the past 15 years, hundreds of bias mitigation methods have been proposed in the pursuit of fairness in machine learning (ML). However, algorithmic biases are domain-, task-, and model-specific, leading to a `portability trap': bias mitigation solutions in one context may not be appropriate in another. Thus, a myriad of design choices have to be made when creating a bias mitigation method, such as the formalization of fairness it pursues, and where and how it intervenes in the ML pipeline. This creates challenges in benchmarking and comparing the relative merits of different bias mitigation methods, and limits their uptake by practitioners. We propose BiMi Sheets as a portable, uniform guide to document the design choices of any bias mitigation method. This enables researchers and practitioners to quickly learn its main characteristics and to compare with their desiderata. Furthermore, the sheets' structure allow for the creation of a structured database of bias mitigation methods. In order to foster the sheets' adoption, we provide a platform for finding and creating BiMi Sheets at bimisheet.com.