Proposed Guidelines for the Responsible Use of Explainable Machine Learning
This work tackles the problem of ensuring ethical and secure deployment of XAI for stakeholders in industries like financial services, though it is incremental as it builds on existing discussions and practices.
The paper addresses the potential misuse of explainable machine learning (XAI) as a faulty safeguard for harmful black-box models, such as in fairwashing or data theft, and proposes guidelines for responsible use, concluding with an argument for employing interpretable models and testing methods in critical systems.
Explainable machine learning (ML) enables human learning from ML, human appeal of automated model decisions, regulatory compliance, and security audits of ML models. Explainable ML (i.e. explainable artificial intelligence or XAI) has been implemented in numerous open source and commercial packages and explainable ML is also an important, mandatory, or embedded aspect of commercial predictive modeling in industries like financial services. However, like many technologies, explainable ML can be misused, particularly as a faulty safeguard for harmful black-boxes, e.g. fairwashing or scaffolding, and for other malevolent purposes like stealing models and sensitive training data. To promote best-practice discussions for this already in-flight technology, this short text presents internal definitions and a few examples before covering the proposed guidelines. This text concludes with a seemingly natural argument for the use of interpretable models and explanatory, debugging, and disparate impact testing methods in life- or mission-critical ML systems.