REFRESH: Responsible and Efficient Feature Reselection Guided by SHAP Values
This addresses the need for more trustworthy AI models under regulations by enabling efficient feature reselection for responsible AI characteristics, though it is incremental as it builds on existing feature selection and SHAP methods.
The paper tackles the problem of efficiently selecting features for secondary model performance characteristics (like fairness and robustness) after an initial feature selection for a primary objective (like accuracy), introducing a method called REFRESH that uses SHAP values and correlation analysis to approximate model predictions without retraining. Empirical results on three datasets, including a large-scale loan defaulting dataset, show that REFRESH can find alternate models with better characteristics efficiently.
Feature selection is a crucial step in building machine learning models. This process is often achieved with accuracy as an objective, and can be cumbersome and computationally expensive for large-scale datasets. Several additional model performance characteristics such as fairness and robustness are of importance for model development. As regulations are driving the need for more trustworthy models, deployed models need to be corrected for model characteristics associated with responsible artificial intelligence. When feature selection is done with respect to one model performance characteristic (eg. accuracy), feature selection with secondary model performance characteristics (eg. fairness and robustness) as objectives would require going through the computationally expensive selection process from scratch. In this paper, we introduce the problem of feature \emph{reselection}, so that features can be selected with respect to secondary model performance characteristics efficiently even after a feature selection process has been done with respect to a primary objective. To address this problem, we propose REFRESH, a method to reselect features so that additional constraints that are desirable towards model performance can be achieved without having to train several new models. REFRESH's underlying algorithm is a novel technique using SHAP values and correlation analysis that can approximate for the predictions of a model without having to train these models. Empirical evaluations on three datasets, including a large-scale loan defaulting dataset show that REFRESH can help find alternate models with better model characteristics efficiently. We also discuss the need for reselection and REFRESH based on regulation desiderata.