Opt-In Art: Learning Art Styles Only from Few Examples
This work addresses the challenge of efficient and controlled artistic style generation for AI and creative applications, offering an incremental improvement by reducing data requirements.
The paper tackles the problem of learning artistic styles from few examples without pre-training on paintings, showing that a model trained only on photographs can adapt to artistic styles and perform on par with state-of-the-art models trained on large artistic datasets.
We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, without access to any painting-related content. We show that it is possible to adapt a model that is trained without paintings to an artistic style, given only few examples. User studies and automatic evaluations confirm that our model (post-adaptation) performs on par with state-of-the-art models trained on massive datasets that contain artistic content like paintings, drawings or illustrations. Finally, using data attribution techniques, we analyze how both artistic and non-artistic datasets contribute to generating artistic-style images. Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data, indicating that artistic style generation can occur in a controlled, opt-in manner using only a limited, carefully selected set of training examples.