An Empirical Study of Accuracy, Fairness, Explainability, Distributional Robustness, and Adversarial Robustness
This work addresses the need for comprehensive evaluation of AI models to ensure trust, though it is incremental as it empirically compares existing metrics without introducing new methods.
The study evaluated AI models across multiple dimensions including accuracy, fairness, explainability, distributional robustness, and adversarial robustness, finding that no single model type excels in all areas and highlighting trade-offs in model selection.
To ensure trust in AI models, it is becoming increasingly apparent that evaluation of models must be extended beyond traditional performance metrics, like accuracy, to other dimensions, such as fairness, explainability, adversarial robustness, and distribution shift. We describe an empirical study to evaluate multiple model types on various metrics along these dimensions on several datasets. Our results show that no particular model type performs well on all dimensions, and demonstrate the kinds of trade-offs involved in selecting models evaluated along multiple dimensions.