Balanced Filtering via Disclosure-Controlled Proxies
This addresses fairness and privacy concerns in machine learning deployment for applications where sensitive attributes must be protected, though it is incremental as it builds on existing proxy and disclosure control techniques.
The paper tackles the problem of collecting balanced cohorts with respect to sensitive groups when group membership cannot be used at deployment, by developing a proxy-based filtering method that controls information disclosure about group membership. The method achieves this with sample- and oracle-efficient training and is experimentally evaluated for generalization.
We study the problem of collecting a cohort or set that is balanced with respect to sensitive groups when group membership is unavailable or prohibited from use at deployment time. Specifically, our deployment-time collection mechanism does not reveal significantly more about the group membership of any individual sample than can be ascertained from base rates alone. To do this, we study a learner that can use a small set of labeled data to train a proxy function that can later be used for this filtering or selection task. We then associate the range of the proxy function with sampling probabilities; given a new example, we classify it using our proxy function and then select it with probability corresponding to its proxy classification. Importantly, we require that the proxy classification does not reveal significantly more information about the sensitive group membership of any individual example compared to population base rates alone (i.e., the level of disclosure should be controlled) and show that we can find such a proxy in a sample- and oracle-efficient manner. Finally, we experimentally evaluate our algorithm and analyze its generalization properties.