Under the Radar -- Auditing Fairness in ML for Humanitarian Mapping
This work addresses fairness concerns in humanitarian mapping for policy-makers, highlighting potential biases that could compromise aid distribution, though it is incremental in applying existing fairness methods to a new domain.
The study investigated bias in humanitarian mapping algorithms for poverty and electricity predictions in Indian villages, finding systematic overestimation of poverty and underestimation of electricity for scheduled tribes, with opposite effects for scheduled castes.
Humanitarian mapping from space with machine learning helps policy-makers to timely and accurately identify people in need. However, recent concerns around fairness and transparency of algorithmic decision-making are a significant obstacle for applying these methods in practice. In this paper, we study if humanitarian mapping approaches from space are prone to bias in their predictions. We map village-level poverty and electricity rates in India based on nighttime lights (NTLs) with linear regression and random forest and analyze if the predictions systematically show prejudice against scheduled caste or tribe communities. To achieve this, we design a causal approach to measure counterfactual fairness based on propensity score matching. This allows to compare villages within a community of interest to synthetic counterfactuals. Our findings indicate that poverty is systematically overestimated and electricity systematically underestimated for scheduled tribes in comparison to a synthetic counterfactual group of villages. The effects have the opposite direction for scheduled castes where poverty is underestimated and electrification overestimated. These results are a warning sign for a variety of applications in humanitarian mapping where fairness issues would compromise policy goals.