Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
This addresses transparency challenges in workplace AI systems for workers and organizations, but it is incremental as it builds on existing literature without introducing new methods or data.
The paper tackles the problem of transparency in enterprise AI knowledge systems, which can cause disproportionate harms by influencing workers' perceptions due to inaccuracies in knowledge extraction and surfacing, and it presents a reflective analysis that formulates transparency as a mediator in shaping different ways of seeing and identifies three necessary dimensions.
Knowledge can't be disentangled from people. As AI knowledge systems mine vast volumes of work-related data, the knowledge that's being extracted and surfaced is intrinsically linked to the people who create and use it. When predictive algorithms that learn from data are used to link knowledge and people, inaccuracies in knowledge extraction and surfacing can lead to disproportionate harms, influencing how individuals see each other and how they see themselves at work. In this paper, we present a reflective analysis of transparency requirements and impacts in this type of systems. We conduct a multidisciplinary literature review to understand the impacts of transparency in workplace settings, introducing the looking-glass metaphor to conceptualize AI knowledge systems as systems that reflect and distort, expanding our view on transparency requirements, implications and challenges. We formulate transparency as a key mediator in shaping different ways of seeing, including seeing into the system, which unveils its capabilities, limitations and behavior, and seeing through the system, which shapes workers' perceptions of their own contributions and others within the organization. Recognizing the sociotechnical nature of these systems, we identify three transparency dimensions necessary to realize the value of AI knowledge systems, namely system transparency, procedural transparency and transparency of outcomes. We discuss key challenges hindering the implementation of these forms of transparency, bringing to light the wider sociotechnical gap and highlighting directions for future Computer-supported Cooperative Work (CSCW) research.