Interpretable Differencing of Machine Learning Models
This addresses the need for interpretable comparisons in scenarios like model selection or updates, offering a domain-specific tool for ML practitioners.
The paper tackles the problem of understanding differences between machine learning models by formalizing model differencing to predict a dissimilarity function with human-interpretable representations, resulting in a Joint Surrogate Tree that provides concise contextual differencing with no loss in fidelity compared to naive approaches.
Understanding the differences between machine learning (ML) models is of interest in scenarios ranging from choosing amongst a set of competing models, to updating a deployed model with new training data. In these cases, we wish to go beyond differences in overall metrics such as accuracy to identify where in the feature space do the differences occur. We formalize this problem of model differencing as one of predicting a dissimilarity function of two ML models' outputs, subject to the representation of the differences being human-interpretable. Our solution is to learn a Joint Surrogate Tree (JST), which is composed of two conjoined decision tree surrogates for the two models. A JST provides an intuitive representation of differences and places the changes in the context of the models' decision logic. Context is important as it helps users to map differences to an underlying mental model of an AI system. We also propose a refinement procedure to increase the precision of a JST. We demonstrate, through an empirical evaluation, that such contextual differencing is concise and can be achieved with no loss in fidelity over naive approaches.