Explainability Requires Interactivity
This addresses the need for more reliable and comprehensive explainability in AI, particularly for vision models, to prevent user misinterpretation and potential risks, representing an incremental improvement over existing static approaches.
The paper tackles the problem of overly simplistic and misleading explanations for deep neural network decisions in computer vision by introducing an interactive framework that allows exhaustive inspection and testing of model decisions, showing that static methods can lead users astray with severe consequences.
When explaining the decisions of deep neural networks, simple stories are tempting but dangerous. Especially in computer vision, the most popular explanation approaches give a false sense of comprehension to its users and provide an overly simplistic picture. We introduce an interactive framework to understand the highly complex decision boundaries of modern vision models. It allows the user to exhaustively inspect, probe, and test a network's decisions. Across a range of case studies, we compare the power of our interactive approach to static explanation methods, showing how these can lead a user astray, with potentially severe consequences.