Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models
This addresses a limitation in text-to-image models for artists and creators, but it is incremental as it builds on existing critiques of data biases.
The paper analyzed the language style of training data for text-to-image diffusion models, finding it lacks figurative speech and subjective elements, which limits the models' ability to generate artistic images without explicit descriptions. It suggests incorporating subjective information into training to enhance creative generation.
The impressive capacity shown by recent text-to-image diffusion models to generate high-quality pictures from textual input prompts has leveraged the debate about the very definition of art. Nonetheless, these models have been trained using text data collected from content-based labelling protocols that focus on describing the items and actions in an image but neglect any subjective appraisal. Consequently, these automatic systems need rigorous descriptions of the elements and the pictorial style of the image to be generated, otherwise failing to deliver. As potential indicators of the actual artistic capabilities of current generative models, we characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models. Considering the sharp difference observed between their language style and that typically employed in artistic contexts, we suggest generative models should incorporate additional sources of subjective information in their training in order to overcome (or at least to alleviate) some of their current limitations, thus effectively unleashing a truly artistic and creative generation.