Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff
This work addresses the problem of slow adoption of machine learning in healthcare due to transparency issues, though it is incremental by applying existing explainability techniques to a specific domain.
The paper tackles the lack of model transparency in healthcare by exploring explainability methods for clinical text classification, specifically mortality prediction using MIMIC-III data, and introduces a framework with metrics like infidelity and local Lipschitz to evaluate the tradeoff between predictive performance and explanation quality.
The healthcare domain is one of the most exciting application areas for machine learning, but a lack of model transparency contributes to a lag in adoption within the industry. In this work, we explore the current art of explainability and interpretability within a case study in clinical text classification, using a task of mortality prediction within MIMIC-III clinical notes. We demonstrate various visualization techniques for fully interpretable methods as well as model-agnostic post hoc attributions, and we provide a generalized method for evaluating the quality of explanations using infidelity and local Lipschitz across model types from logistic regression to BERT variants. With these metrics, we introduce a framework through which practitioners and researchers can assess the frontier between a model's predictive performance and the quality of its available explanations. We make our code available to encourage continued refinement of these methods.