Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model
This addresses the problem of evaluating trust in XAI for medical experts, but it is incremental as it applies an existing method (FCMs) to a new context.
The study tackled measuring perceived trust in XAI-assisted decision-making by eliciting medical experts' mental models using Fuzzy Cognitive Maps, and found that quantified values from the method could determine whether experts trust or distrust the XAI model, with analysis comparing these values to diagnostic task performance.
This empirical study proposes a novel methodology to measure users' perceived trust in an Explainable Artificial Intelligence (XAI) model. To do so, users' mental models are elicited using Fuzzy Cognitive Maps (FCMs). First, we exploit an interpretable Machine Learning (ML) model to classify suspected COVID-19 patients into positive or negative cases. Then, Medical Experts' (MEs) conduct a diagnostic decision-making task based on their knowledge and then prediction and interpretations provided by the XAI model. In order to evaluate the impact of interpretations on perceived trust, explanation satisfaction attributes are rated by MEs through a survey. Then, they are considered as FCM's concepts to determine their influences on each other and, ultimately, on the perceived trust. Moreover, to consider MEs' mental subjectivity, fuzzy linguistic variables are used to determine the strength of influences. After reaching the steady state of FCMs, a quantified value is obtained to measure the perceived trust of each ME. The results show that the quantified values can determine whether MEs trust or distrust the XAI model. We analyze this behavior by comparing the quantified values with MEs' performance in completing diagnostic tasks.