Training Data Attribution for Image Generation using Ontology-Aligned Knowledge Graphs
This addresses transparency and copyright concerns for users of generative AI models, though it appears incremental as it adapts existing KG techniques to visual data.
The paper tackles the problem of attributing generative model outputs to specific training data by introducing a framework that constructs ontology-aligned knowledge graphs from images using multimodal LLMs, enabling tracing of influences for copyright analysis and transparency. It validates the method through experiments on locally trained models and large-scale models, though no concrete numerical results are provided.
As generative models become powerful, concerns around transparency, accountability, and copyright violations have intensified. Understanding how specific training data contributes to a model's output is critical. We introduce a framework for interpreting generative outputs through the automatic construction of ontologyaligned knowledge graphs (KGs). While automatic KG construction from natural text has advanced, extracting structured and ontology-consistent representations from visual content remains challenging -- due to the richness and multi-object nature of images. Leveraging multimodal large language models (LLMs), our method extracts structured triples from images, aligned with a domain-specific ontology. By comparing the KGs of generated and training images, we can trace potential influences, enabling copyright analysis, dataset transparency, and interpretable AI. We validate our method through experiments on locally trained models via unlearning, and on large-scale models through a style-specific experiment. Our framework supports the development of AI systems that foster human collaboration, creativity and stimulate curiosity.