The Malicious Technical Ecosystem: Exposing Limitations in Technical Governance of AI-Generated Non-Consensual Intimate Images of Adults
This work addresses the problem of AI-generated non-consensual intimate images for adult survivors, highlighting governance gaps in a critical societal issue.
The paper identifies a 'malicious technical ecosystem' of open-source face-swapping models and nearly 200 nudifying software programs that enable non-technical users to create AI-generated non-consensual intimate images of adults within minutes, and shows that current governance methods, as reflected in the NIST AI 100-4 report, fail to effectively regulate this ecosystem.
In this paper, we adopt a survivor-centered approach to locate and dissect the role of sociotechnical AI governance in preventing AI-Generated Non-Consensual Intimate Images (AIG-NCII) of adults, colloquially known as "deep fake pornography." We identify a "malicious technical ecosystem" or "MTE," comprising of open-source face-swapping models and nearly 200 "nudifying" software programs that allow non-technical users to create AIG-NCII within minutes. Then, using the National Institute of Standards and Technology (NIST) AI 100-4 report as a reflection of current synthetic content governance methods, we show how the current landscape of practices fails to effectively regulate the MTE for adult AIG-NCII, as well as flawed assumptions explaining these gaps.