Margin-distancing for safe model explanation
This addresses the problem of making model explanations safe from gaming in consequential settings, representing a technical study of an issue previously debated mainly in legal literature.
The paper tackles the tension between transparency and vulnerability to gaming in machine learning model explanations by proposing a formulation and tradeoff method, with empirical results on real-world datasets.
The growing use of machine learning models in consequential settings has highlighted an important and seemingly irreconcilable tension between transparency and vulnerability to gaming. While this has sparked sizable debate in legal literature, there has been comparatively less technical study of this contention. In this work, we propose a clean-cut formulation of this tension and a way to make the tradeoff between transparency and gaming. We identify the source of gaming as being points close to the \emph{decision boundary} of the model. And we initiate an investigation on how to provide example-based explanations that are expansive and yet consistent with a version space that is sufficiently uncertain with respect to the boundary points' labels. Finally, we furnish our theoretical results with empirical investigations of this tradeoff on real-world datasets.