A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms
This addresses the need for more inclusive AI transparency for non-technical stakeholders, but it is incremental as it focuses on study setup rather than results.
The paper tackles the problem that current AI transparency mechanisms primarily serve technical stakeholders, by proposing a large-scale, mixed-methods user study to evaluate how well these mechanisms work for a diverse set of stakeholders in industries like healthcare or criminal justice.
Given that there are a variety of stakeholders involved in, and affected by, decisions from machine learning (ML) models, it is important to consider that different stakeholders have different transparency needs. Previous work found that the majority of deployed transparency mechanisms primarily serve technical stakeholders. In our work, we want to investigate how well transparency mechanisms might work in practice for a more diverse set of stakeholders by conducting a large-scale, mixed-methods user study across a range of organizations, within a particular industry such as health care, criminal justice, or content moderation. In this paper, we outline the setup for our study.