Ewart J. de Visser

Matouš Jelínek, Nadine Schlicker, Ewart de Visser

Calibrated trust in automated systems (Lee and See 2004) is critical for their safe and seamless integration into society. Users should only rely on a system recommendation when it is actually correct and reject it when it is factually wrong. One requirement to achieve this goal is an accurate trustworthiness assessment, ensuring that the user's perception of the system's trustworthiness aligns with its actual trustworthiness, allowing users to make informed decisions about the extent to which they can rely on the system (Schlicker et al. 2022). We propose six design guidelines to help designers optimize for accurate trustworthiness assessments, thus fostering ethical and responsible human-automation interactions. The proposed guidelines are derived from existing literature in various fields, such as human-computer interaction, cognitive psychology, automation research, user-experience design, and ethics. We are incorporating key principles from the field of pragmatics, specifically the cultivation of common ground (H. H. Clark 1996) and Gricean communication maxims (Grice 1975). These principles are essential for the design of automated systems because the user's perception of the system's trustworthiness is shaped by both environmental contexts, such as organizational culture or societal norms, and by situational context, including the specific circumstances or scenarios in which the interaction occurs (Hoff and Bashir 2015). Our proposed guidelines provide actionable insights for designers to create automated systems that make relevant trustworthiness cues available. This would ideally foster calibrated trust and more satisfactory, productive, and safe interactions between humans and automated systems. Furthermore, the proposed heuristics might work as a tool for evaluating to what extent existing systems enable users to accurately assess a system's trustworthiness.

Andres Rosero, Faustina Dinh, Ewart J. de Visser et al.

This paper describes a research study that aims to investigate changes in effective communication during human-AI collaboration with special attention to the perception of competence among team members and varying levels of task load placed on the team. We will also investigate differences between human-human teamwork and human-agent teamwork. Our project will measure differences in the communication quality, team perception and performance of a human actor playing a Commercial Off - The Shelf game (COTS) with either a human teammate or a simulated AI teammate under varying task load. We argue that the increased cognitive workload associated with increases task load will be negatively associated with team performance and have a negative impact on communication quality. In addition, we argue that positive team perceptions will have a positive impact on the communication quality between a user and teammate in both the human and AI teammate conditions. This project will offer more refined insights on Human - AI relationship dynamics in collaborative tasks by considering communication quality, team perception, and performance under increasing cognitive workload.