Subverting machines, fluctuating identities: Re-learning human categorization
This work addresses the problem of rigid identity categorization in AI systems, which can perpetuate biases and fairness issues, but it is incremental as it primarily introduces a theoretical framework without empirical validation.
The paper critiques the default paradigm in AI that treats human identity as static and discrete, proposing instead a theory of identity as autopoiesis (malleable and constructed through interaction) to address how this paradigm reinforces power differentials and limits fairness practices. It sketches approaches like multilevel optimization and relational learning to enable machines to express identity as perpetually in flux, though it raises open questions without concrete numerical results.
Most machine learning systems that interact with humans construct some notion of a person's "identity," yet the default paradigm in AI research envisions identity with essential attributes that are discrete and static. In stark contrast, strands of thought within critical theory present a conception of identity as malleable and constructed entirely through interaction; a doing rather than a being. In this work, we distill some of these ideas for machine learning practitioners and introduce a theory of identity as autopoiesis, circular processes of formation and function. We argue that the default paradigm of identity used by the field immobilizes existing identity categories and the power differentials that co$\unicode{x2010}$occur, due to the absence of iterative feedback to our models. This includes a critique of emergent AI fairness practices that continue to impose the default paradigm. Finally, we apply our theory to sketch approaches to autopoietic identity through multilevel optimization and relational learning. While these ideas raise many open questions, we imagine the possibilities of machines that are capable of expressing human identity as a relationship perpetually in flux.