Uncertainty-aware predictive modeling for fair data-driven decisions
This work addresses the safety gap in fair ML systems for socio-technical applications like job profiling, though it appears incremental by building on existing uncertainty and fairness literature.
The paper tackles the problem of ensuring safety in fair machine learning systems by integrating uncertainty awareness, proposing that fair models must account for uncertainty to enable safe fail options for uncertain categorizations. It introduces semi-structured deep distributional regression as a framework and demonstrates its application in algorithmic profiling of job seekers.
Both industry and academia have made considerable progress in developing trustworthy and responsible machine learning (ML) systems. While critical concepts like fairness and explainability are often addressed, the safety of systems is typically not sufficiently taken into account. By viewing data-driven decision systems as socio-technical systems, we draw on the uncertainty in ML literature to show how fairML systems can also be safeML systems. We posit that a fair model needs to be an uncertainty-aware model, e.g. by drawing on distributional regression. For fair decisions, we argue that a safe fail option should be used for individuals with uncertain categorization. We introduce semi-structured deep distributional regression as a modeling framework which addresses multiple concerns brought against standard ML models and show its use in a real-world example of algorithmic profiling of job seekers.