Verifiable Fairness: Privacy-preserving Computation of Fairness for Machine Learning Systems
This addresses the need for trustworthy fairness audits in ML systems, particularly for auditors and stakeholders, though it is incremental as it builds on existing cryptographic techniques.
The paper tackles the problem of verifying fairness in machine learning systems while preserving privacy, proposing Fairness as a Service (FaaS) as a secure, verifiable, and privacy-preserving protocol that supports various fairness metrics and was successfully implemented on a dataset with thousands of entries.
Fair machine learning is a thriving and vibrant research topic. In this paper, we propose Fairness as a Service (FaaS), a secure, verifiable and privacy-preserving protocol to computes and verify the fairness of any machine learning (ML) model. In the deisgn of FaaS, the data and outcomes are represented through cryptograms to ensure privacy. Also, zero knowledge proofs guarantee the well-formedness of the cryptograms and underlying data. FaaS is model--agnostic and can support various fairness metrics; hence, it can be used as a service to audit the fairness of any ML model. Our solution requires no trusted third party or private channels for the computation of the fairness metric. The security guarantees and commitments are implemented in a way that every step is securely transparent and verifiable from the start to the end of the process. The cryptograms of all input data are publicly available for everyone, e.g., auditors, social activists and experts, to verify the correctness of the process. We implemented FaaS to investigate performance and demonstrate the successful use of FaaS for a publicly available data set with thousands of entries.