Explaining Deep Neural Networks
This addresses the need for interpretability in AI systems for users in high-stakes domains, but it appears incremental as it reviews existing directions without presenting new results.
The paper tackles the problem of deep neural networks being uninterpretable, which is critical in domains like healthcare and finance, by investigating two explanation directions: feature-based post-hoc methods and self-explanatory models that generate natural language explanations.
Deep neural networks are becoming more and more popular due to their revolutionary success in diverse areas, such as computer vision, natural language processing, and speech recognition. However, the decision-making processes of these models are generally not interpretable to users. In various domains, such as healthcare, finance, or law, it is critical to know the reasons behind a decision made by an artificial intelligence system. Therefore, several directions for explaining neural models have recently been explored. In this thesis, I investigate two major directions for explaining deep neural networks. The first direction consists of feature-based post-hoc explanatory methods, that is, methods that aim to explain an already trained and fixed model (post-hoc), and that provide explanations in terms of input features, such as tokens for text and superpixels for images (feature-based). The second direction consists of self-explanatory neural models that generate natural language explanations, that is, models that have a built-in module that generates explanations for the predictions of the model.