Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
It highlights how data annotation is an exercise of power with societal implications, offering a broader view beyond individual subjectivity for researchers and practitioners in computer vision.
This paper investigates power dynamics in industrial image data annotation, revealing that annotators' work is shaped by the interests and priorities of higher-level actors, leading to arbitrary classifications being imposed on data.
The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators subjectivity as a major cause for biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.