A Framework for Critical Evaluation of Text-to-Image Models: Integrating Art Historical Analysis, Artistic Exploration, and Critical Prompt Engineering
This addresses the need for better evaluation methods in AI for researchers and practitioners, though it is incremental as it builds on existing bias studies and technical metrics.
The paper tackles the problem of evaluating text-to-image models by proposing an interdisciplinary framework that integrates art historical analysis, artistic exploration, and critical prompt engineering, resulting in a more nuanced understanding of their capabilities and societal implications, as demonstrated through case studies revealing biases related to gender, race, and cultural representation.
This paper proposes a novel interdisciplinary framework for the critical evaluation of text-to-image models, addressing the limitations of current technical metrics and bias studies. By integrating art historical analysis, artistic exploration, and critical prompt engineering, the framework offers a more nuanced understanding of these models' capabilities and societal implications. Art historical analysis provides a structured approach to examine visual and symbolic elements, revealing potential biases and misrepresentations. Artistic exploration, through creative experimentation, uncovers hidden potentials and limitations, prompting critical reflection on the algorithms' assumptions. Critical prompt engineering actively challenges the model's assumptions, exposing embedded biases. Case studies demonstrate the framework's practical application, showcasing how it can reveal biases related to gender, race, and cultural representation. This comprehensive approach not only enhances the evaluation of text-to-image models but also contributes to the development of more equitable, responsible, and culturally aware AI systems.