Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors
This addresses the problem of creating emotionally compelling images from text for social media users, representing an incremental advancement by extending prior methods with generative priors.
The paper tackles the Affective Image Filter (AIF) task to reflect emotions from text into images, proposing AIF-B and AIF-D models that achieve superior performance in content consistency and emotional fidelity compared to state-of-the-art methods, with user studies showing they are significantly more effective at evoking specific emotions.
Social media platforms enable users to express emotions by posting text with accompanying images. In this paper, we propose the Affective Image Filter (AIF) task, which aims to reflect visually-abstract emotions from text into visually-concrete images, thereby creating emotionally compelling results. We first introduce the AIF dataset and the formulation of the AIF models. Then, we present AIF-B as an initial attempt based on a multi-modal transformer architecture. After that, we propose AIF-D as an extension of AIF-B towards deeper emotional reflection, effectively leveraging generative priors from pre-trained large-scale diffusion models. Quantitative and qualitative experiments demonstrate that AIF models achieve superior performance for both content consistency and emotional fidelity compared to state-of-the-art methods. Extensive user study experiments demonstrate that AIF models are significantly more effective at evoking specific emotions. Based on the presented results, we comprehensively discuss the value and potential of AIF models.