Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
This addresses the challenge of translating interpretability insights into actionable model edits for domain experts and data scientists, though it is incremental as it builds on existing interpretability methods.
The researchers tackled the problem of acting on undesirable patterns revealed by interpretability techniques by developing GAM Changer, an interactive system that allows domain experts and data scientists to edit Generalized Additive Models to align them with human knowledge and values, with evaluations showing it is easy to use and fits into workflows.
Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collaboration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action--empowering users to analyze, validate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneumonia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users' web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the following public demo link: https://interpret.ml/gam-changer.