Coincidental Generation
This addresses a privacy and legal problem for organizations using generative AI, such as in virtual modeling or stock photography, by highlighting an inherent risk in current models.
The paper identifies a new privacy concern called coincidental generation, where generative AI models produce outputs that coincidentally resemble real entities not in the training data, leading to risks like misappropriation of likeness and legal exposure for organizations.
Generative A.I. models have emerged as versatile tools across diverse industries, with applications in privacy-preserving data sharing, computational art, personalization of products and services, and immersive entertainment. Here, we introduce a new privacy concern in the adoption and use of generative A.I. models: that of coincidental generation, where a generative model's output is similar enough to an existing entity, beyond those represented in the dataset used to train the model, to be mistaken for it. Consider, for example, synthetic portrait generators, which are today deployed in commercial applications such as virtual modeling agencies and synthetic stock photography. Due to the low intrinsic dimensionality of human face perception, every synthetically generated face will coincidentally resemble an actual person. Such examples of coincidental generation all but guarantee the misappropriation of likeness and expose organizations that use generative A.I. to legal and regulatory risk.