Does Faithfulness Conflict with Plausibility? An Empirical Study in Explainable AI across NLP Tasks
This addresses the trade-off between accurate model explanations and expert consistency in NLP, with incremental insights for algorithm design.
The study investigated whether faithfulness and plausibility conflict in explainable AI for NLP tasks, finding that perturbation-based methods like Shapley value and LIME can achieve high levels of both dimensions.
Explainability algorithms aimed at interpreting decision-making AI systems usually consider balancing two critical dimensions: 1) \textit{faithfulness}, where explanations accurately reflect the model's inference process. 2) \textit{plausibility}, where explanations are consistent with domain experts. However, the question arises: do faithfulness and plausibility inherently conflict? In this study, through a comprehensive quantitative comparison between the explanations from the selected explainability methods and expert-level interpretations across three NLP tasks: sentiment analysis, intent detection, and topic labeling, we demonstrate that traditional perturbation-based methods Shapley value and LIME could attain greater faithfulness and plausibility. Our findings suggest that rather than optimizing for one dimension at the expense of the other, we could seek to optimize explainability algorithms with dual objectives to achieve high levels of accuracy and user accessibility in their explanations.