RuleMatrix: Visualizing and Understanding Classifiers with Rules
This work addresses the need for transparency in ML systems for non-expert users, representing an incremental improvement in interpretability tools.
The paper tackles the problem of making machine learning classifiers interpretable for domain experts without ML knowledge by introducing RuleMatrix, a visualization technique that extracts rule-based representations from black-box models, and evaluates its effectiveness through use cases and a usability study.
With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. We design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.