Exclusion and Inclusion -- A model agnostic approach to feature importance in DNNs
This addresses the interpretability bottleneck for real-world DNN applications in NLP, though it appears incremental as it builds on existing feature importance methods.
The authors tackled the problem of DNNs being black boxes by introducing a model-agnostic algorithm for phrase-wise feature importance, achieving robustness to outliers and demonstrating generalizability across regression and classification tasks.
Deep Neural Networks in NLP have enabled systems to learn complex non-linear relationships. One of the major bottlenecks towards being able to use DNNs for real world applications is their characterization as black boxes. To solve this problem, we introduce a model agnostic algorithm which calculates phrase-wise importance of input features. We contend that our method is generalizable to a diverse set of tasks, by carrying out experiments for both Regression and Classification. We also observe that our approach is robust to outliers, implying that it only captures the essential aspects of the input.