TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning
This work addresses the need for trustworthy and interpretable AI in clinical decision support, offering a scalable solution for domain experts, though it is incremental as it builds on existing methods like BERT and LIME.
The paper tackled the problem of clinical language models providing untrustworthy predictions and explanations for lengthy electronic health records by introducing TT-XAI, a framework that uses keyword distillation and LLM reasoning, resulting in improved classification performance and interpretability as confirmed by evaluations including a blinded human study.
Clinical language models often struggle to provide trustworthy predictions and explanations when applied to lengthy, unstructured electronic health records (EHRs). This work introduces TT-XAI, a lightweight and effective framework that improves both classification performance and interpretability through domain-aware keyword distillation and reasoning with large language models (LLMs). First, we demonstrate that distilling raw discharge notes into concise keyword representations significantly enhances BERT classifier performance and improves local explanation fidelity via a focused variant of LIME. Second, we generate chain-of-thought clinical explanations using keyword-guided prompts to steer LLMs, producing more concise and clinically relevant reasoning. We evaluate explanation quality using deletion-based fidelity metrics, self-assessment via LLaMA-3 scoring, and a blinded human study with domain experts. All evaluation modalities consistently favor the keyword-augmented method, confirming that distillation enhances both machine and human interpretability. TT-XAI offers a scalable pathway toward trustworthy, auditable AI in clinical decision support.