Model Cards for AI Teammates: Comparing Human-AI Team Familiarization Methods for High-Stakes Environments
This addresses the problem of optimizing human-AI team performance in critical settings like surveillance, offering practical insights for interface design, though it is incremental as it builds on existing human-AI interaction research.
The study compared three methods for familiarizing humans with AI teammates in a high-stakes ISR environment, finding that documentation-based familiarization led to faster strategy adoption but risk-averse behavior, while direct interaction allowed more risk-taking but weaker understanding of the AI's internal processes.
We compare three methods of familiarizing a human with an artificial intelligence (AI) teammate ("agent") prior to operation in a collaborative, fast-paced intelligence, surveillance, and reconnaissance (ISR) environment. In a between-subjects user study (n=60), participants either read documentation about the agent, trained alongside the agent prior to the mission, or were given no familiarization. Results showed that the most valuable information about the agent included details of its decision-making algorithms and its relative strengths and weaknesses compared to the human. This information allowed the familiarization groups to form sophisticated team strategies more quickly than the control group. Documentation-based familiarization led to the fastest adoption of these strategies, but also biased participants towards risk-averse behavior that prevented high scores. Participants familiarized through direct interaction were able to infer much of the same information through observation, and were more willing to take risks and experiment with different control modes, but reported weaker understanding of the agent's internal processes. Significant differences were seen between individual participants' risk tolerance and methods of AI interaction, which should be considered when designing human-AI control interfaces. Based on our findings, we recommend a human-AI team familiarization method that combines AI documentation, structured in-situ training, and exploratory interaction.