Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise
This work addresses the challenge of measuring and understanding trust in AI systems for users and developers, but it is incremental as a pilot study with limited participants.
The study investigated how domain expertise influences trust in prediction models, finding that expertise alone does not fully predict trust perceptions and identifying six themes affecting trust evolution.
People's trust in prediction models can be affected by many factors, including domain expertise like knowledge about the application domain and experience with predictive modelling. However, to what extent and why domain expertise impacts people's trust is not entirely clear. In addition, accurately measuring people's trust remains challenging. We share our results and experiences of an exploratory pilot study in which four people experienced with predictive modelling systematically explore a visual analytics system with an unknown prediction model. Through a mixed-methods approach involving Likert-type questions and a semi-structured interview, we investigate how people's trust evolves during their exploration, and we distil six themes that affect their trust in the prediction model. Our results underline the multi-faceted nature of trust, and suggest that domain expertise alone cannot fully predict people's trust perceptions.