Interpretability via Model Extraction
This addresses the need for interpretability in ML models used for consequential decisions, offering a method to understand and debug them, though it is incremental as it builds on existing approximation techniques.
The paper tackles the problem of interpreting complex blackbox machine learning models by proposing model extraction, which approximates them with interpretable models to reflect their statistical properties, demonstrating its application on random forests, neural nets, and reinforcement learning policies from UCI datasets and classical problems.
The ability to interpret machine learning models has become increasingly important now that machine learning is used to inform consequential decisions. We propose an approach called model extraction for interpreting complex, blackbox models. Our approach approximates the complex model using a much more interpretable model; as long as the approximation quality is good, then statistical properties of the complex model are reflected in the interpretable model. We show how model extraction can be used to understand and debug random forests and neural nets trained on several datasets from the UCI Machine Learning Repository, as well as control policies learned for several classical reinforcement learning problems.