Multimodal LLMs Can Reason about Aesthetics in Zero-Shot
This work addresses the problem of AI lacking genuine artistic impact for applications like art tutoring and image generation, though it is incremental in refining existing MLLM capabilities.
The paper tackles the challenge of enabling AI to perform sophisticated aesthetic judgment beyond visual appeal by investigating Multimodal LLMs (MLLMs) in zero-shot settings, finding that MLLMs tend to hallucinate but can be improved with an evidence-based reasoning process (ArtCoT) to align better with human judgment, achieving significant improvements in alignment.
The rapid technical progress of generative art (GenArt) has democratized the creation of visually appealing imagery. However, achieving genuine artistic impact - the kind that resonates with viewers on a deeper, more meaningful level - remains formidable as it requires a sophisticated aesthetic sensibility. This sensibility involves a multifaceted cognitive process extending beyond mere visual appeal, which is often overlooked by current computational methods. This paper pioneers an approach to capture this complex process by investigating how the reasoning capabilities of Multimodal LLMs (MLLMs) can be effectively elicited to perform aesthetic judgment. Our analysis reveals a critical challenge: MLLMs exhibit a tendency towards hallucinations during aesthetic reasoning, characterized by subjective opinions and unsubstantiated artistic interpretations. We further demonstrate that these hallucinations can be suppressed by employing an evidence-based and objective reasoning process, as substantiated by our proposed baseline, ArtCoT. MLLMs prompted by this principle produce multifaceted, in-depth aesthetic reasoning that aligns significantly better with human judgment. These findings have direct applications in areas such as AI art tutoring and as reward models for image generation. Ultimately, we hope this work paves the way for AI systems that can truly understand, appreciate, and contribute to art that aligns with human aesthetic values. Project homepage: https://github.com/songrise/MLLM4Art.