Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?
This work addresses a fundamental limitation in generative AI's ability to follow simple instructions, which is important for developers seeking more controllable and reliable AI systems.
This paper explores the "Paradox of Simplicity" in generative AI, where models struggle with simple, deterministic tasks despite excelling at complex ones. The authors formalize "Obedience" as alignment with instructions and introduce a hierarchical grading system, along with a new benchmark called VIOLIN for evaluating pure color generation across six variants, revealing fundamental limitations in state-of-the-art models.
Recent advances in generative AI have demonstrated remarkable ability to produce high-quality content. However, these models often exhibit "Paradox of Simplicity": while they can render intricate landscapes, they often fail at simple, deterministic tasks. To address this, we formalize Obedience as the ability to align with instructions and establish a hierarchical grading system ranging from basic semantic alignment to pixel-level systemic precision, which provides a unified paradigm for incorporating and categorizing existing literature. Then, we conduct case studies to identify common obedience gaps, revealing how generative priors often override logical constraints. To evaluate high-level obedience, we present VIOLIN (VIsual Obedience Level-4 EvaluatIoN), the first benchmark focused on pure color generation across six variants. Extensive experiments on SOTA models reveal fundamental obedience limitations and further exploratory insights. By establishing this framework, we aim to draw more attention on AI Obedience and encourage deeper exploration to bridge this gap.