On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness
This addresses the challenge of designing trustworthy AI systems for stakeholders, but it is incremental as it builds on existing psychological findings to refine engineering priorities.
The paper tackles the problem that explainability requirements may not effectively promote trust in software systems, as recent studies show explanations do not necessarily facilitate trust. It argues that engineering for trustworthiness, with explainability as a key contributor, is a more suitable approach, aiming to achieve both trustworthiness and stakeholder trust.
Recently, requirements for the explainability of software systems have gained prominence. One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders' trust in a system. Although this seems intuitively appealing, recent psychological studies indicate that explanations do not necessarily facilitate trust. Thus, explainability requirements might not be suitable for promoting trust. One way to accommodate this finding is, we suggest, to focus on trustworthiness instead of trust. While these two may come apart, we ideally want both: a trustworthy system and the stakeholder's trust. In this paper, we argue that even though trustworthiness does not automatically lead to trust, there are several reasons to engineer primarily for trustworthiness -- and that a system's explainability can crucially contribute to its trustworthiness.