RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation
This work addresses the problem of generating realistic, customizable identity-specific images for applications in digital media and AI art, representing an incremental improvement by combining local and global control mechanisms.
The paper tackles the challenge of identity customization in text-to-image synthesis, where existing methods struggle with small face fidelity, precise control over face attributes, and multi-person customization; the proposed RealisID method achieves scale-robust identity fidelity and fine control over facial details, location, pose, and expression, as demonstrated through extensive experiments.
Recently, the success of text-to-image synthesis has greatly advanced the development of identity customization techniques, whose main goal is to produce realistic identity-specific photographs based on text prompts and reference face images. However, it is difficult for existing identity customization methods to simultaneously meet the various requirements of different real-world applications, including the identity fidelity of small face, the control of face location, pose and expression, as well as the customization of multiple persons. To this end, we propose a scale-robust and fine-controllable method, namely RealisID, which learns different control capabilities through the cooperation between a pair of local and global branches. Specifically, by using cropping and up-sampling operations to filter out face-irrelevant information, the local branch concentrates the fine control of facial details and the scale-robust identity fidelity within the face region. Meanwhile, the global branch manages the overall harmony of the entire image. It also controls the face location by taking the location guidance as input. As a result, RealisID can benefit from the complementarity of these two branches. Finally, by implementing our branches with two different variants of ControlNet, our method can be easily extended to handle multi-person customization, even only trained on single-person datasets. Extensive experiments and ablation studies indicate the effectiveness of RealisID and verify its ability in fulfilling all the requirements mentioned above.