Visual Model Selection using Feature Importance Clusters in Fairness-Performance Similarity Optimized Space
This addresses the problem for stakeholders in algorithmic decision-making who need to navigate trade-offs between fairness and performance, though it is incremental as it builds on existing fair ML methods with a new visualization and clustering approach.
The paper tackles the challenge of selecting fair machine learning models from a diverse portfolio by proposing an interactive framework that clusters models based on fairness-performance similarity and feature importance, enabling stakeholders to explore and choose models aligned with their values.
In the context of algorithmic decision-making, fair machine learning methods often yield multiple models that balance predictive fairness and performance in varying degrees. This diversity introduces a challenge for stakeholders who must select a model that aligns with their specific requirements and values. To address this, we propose an interactive framework that assists in navigating and interpreting the trade-offs across a portfolio of models. Our approach leverages weakly supervised metric learning to learn a Mahalanobis distance that reflects similarity in fairness and performance outcomes, effectively structuring the feature importance space of the models according to stakeholder-relevant criteria. We then apply clustering technique (k-means) to group models based on their transformed representations of feature importances, allowing users to explore clusters of models with similar predictive behaviors and fairness characteristics. This facilitates informed decision-making by helping users understand how models differ not only in their fairness-performance balance but also in the features that drive their predictions.