Towards Localized Fine-Grained Control for Facial Expression Generation
This work addresses the problem of generating authentic and varied facial expressions in AI-generated content, which is incremental as it builds on existing generative models by adding AU-based control.
The paper tackles the challenge of achieving precise and localized control over facial expressions in generative models by proposing the use of action units (AUs) to describe individual facial muscle movements, enabling the generation of nuanced and unconventional expressions beyond typical emotional models.
Generative models have surged in popularity recently due to their ability to produce high-quality images and video. However, steering these models to produce images with specific attributes and precise control remains challenging. Humans, particularly their faces, are central to content generation due to their ability to convey rich expressions and intent. Current generative models mostly generate flat neutral expressions and characterless smiles without authenticity. Other basic expressions like anger are possible, but are limited to the stereotypical expression, while other unconventional facial expressions like doubtful are difficult to reliably generate. In this work, we propose the use of AUs (action units) for facial expression control in face generation. AUs describe individual facial muscle movements based on facial anatomy, allowing precise and localized control over the intensity of facial movements. By combining different action units, we unlock the ability to create unconventional facial expressions that go beyond typical emotional models, enabling nuanced and authentic reactions reflective of real-world expressions. The proposed method can be seamlessly integrated with both text and image prompts using adapters, offering precise and intuitive control of the generated results. Code and dataset are available in {https://github.com/tvaranka/fineface}.