M$^3$Face: A Unified Multi-Modal Multilingual Framework for Human Face Generation and Editing
This work addresses the problem of simplifying face generation and editing for users in computer vision by automating modality creation, though it appears incremental as it builds on existing multi-modal methods.
The authors tackled the challenge of manually creating conditioning modalities for face generation and editing by introducing M3Face, a framework that uses text input to automatically generate controls like semantic segmentation and facial landmarks, resulting in a unified multi-modal multilingual system with demonstrated capabilities through extensive experiments.
Human face generation and editing represent an essential task in the era of computer vision and the digital world. Recent studies have shown remarkable progress in multi-modal face generation and editing, for instance, using face segmentation to guide image generation. However, it may be challenging for some users to create these conditioning modalities manually. Thus, we introduce M3Face, a unified multi-modal multilingual framework for controllable face generation and editing. This framework enables users to utilize only text input to generate controlling modalities automatically, for instance, semantic segmentation or facial landmarks, and subsequently generate face images. We conduct extensive qualitative and quantitative experiments to showcase our frameworks face generation and editing capabilities. Additionally, we propose the M3CelebA Dataset, a large-scale multi-modal and multilingual face dataset containing high-quality images, semantic segmentations, facial landmarks, and different captions for each image in multiple languages. The code and the dataset will be released upon publication.