Certifying Robustness to Programmable Data Bias in Decision Trees
This addresses the challenge of ensuring model fairness and reliability in the face of societal and data biases, particularly for interpretable decision trees, though it is incremental in applying symbolic methods to this specific problem.
The paper tackles the problem of certifying that decision-tree models are robust to programmable data biases, such as missing data for minorities, by using a novel symbolic technique to ensure consistent predictions across potentially infinite biased datasets. It demonstrates viability on fairness datasets with various bias models.
Datasets can be biased due to societal inequities, human biases, under-representation of minorities, etc. Our goal is to certify that models produced by a learning algorithm are pointwise-robust to potential dataset biases. This is a challenging problem: it entails learning models for a large, or even infinite, number of datasets, ensuring that they all produce the same prediction. We focus on decision-tree learning due to the interpretable nature of the models. Our approach allows programmatically specifying bias models across a variety of dimensions (e.g., missing data for minorities), composing types of bias, and targeting bias towards a specific group. To certify robustness, we use a novel symbolic technique to evaluate a decision-tree learner on a large, or infinite, number of datasets, certifying that each and every dataset produces the same prediction for a specific test point. We evaluate our approach on datasets that are commonly used in the fairness literature, and demonstrate our approach's viability on a range of bias models.