Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
This work addresses the challenge of making art more accessible to the general public by providing detailed, knowledgeable descriptions, though it appears incremental in its approach.
The paper tackles the problem of generating comprehensive descriptions for fine-art paintings by addressing multiple aspects like style and content, and incorporating external knowledge, resulting in outstanding results in topic diversity and information veracity as validated through quantitative and qualitative analyses.
Have you ever looked at a painting and wondered what is the story behind it? This work presents a framework to bring art closer to people by generating comprehensive descriptions of fine-art paintings. Generating informative descriptions for artworks, however, is extremely challenging, as it requires to 1) describe multiple aspects of the image such as its style, content, or composition, and 2) provide background and contextual knowledge about the artist, their influences, or the historical period. To address these challenges, we introduce a multi-topic and knowledgeable art description framework, which modules the generated sentences according to three artistic topics and, additionally, enhances each description with external knowledge. The framework is validated through an exhaustive analysis, both quantitative and qualitative, as well as a comparative human evaluation, demonstrating outstanding results in terms of both topic diversity and information veracity.