Prompt fidelity of ChatGPT4o / Dall-E3 text-to-image visualisations
This identifies measurable fidelity gaps in AI image generation, relevant for bias detection and model evaluation, though it is incremental as it applies existing methods to new data.
The study measured how accurately ChatGPT4o/DALL-E3 renders specified attributes in text-to-image visualizations, finding that 15.6% of attributes across datasets were incorrectly rendered, with errors highest for personal attributes like age.
This study examines the prompt fidelity of ChatGPT4o / DALL-E3 text-to-image visualisations by analysing whether attributes explicitly specified in autogenously generated prompts are correctly rendered in the resulting images. Using two public-domain datasets comprising 200 visualisations of women working in the cultural and creative industries and 230 visualisations of museum curators, the study assessed accuracy across personal attributes (age, hair), appearance (attire, glasses), and paraphernalia (name tags, clipboards). While correctly rendered in most cases, DALL-E3 deviated from prompt specifications in 15.6% of all attributes (n=710). Errors were lowest for paraphernalia, moderate for personal appearance, and highest for depictions of the person themselves, particularly age. These findings demonstrate measurable prompt-to-image fidelity gaps with implications for bias detection and model evaluation.