ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation
This work addresses the problem of resource-intensive and inflexible control in face generation for AI and graphics applications, offering a plug-and-play solution that is incremental in its integration of existing models.
The paper tackles the challenge of achieving fine-grained control over facial features in text-to-face generation by proposing ExpertGen, a training-free framework that leverages pre-trained expert models to guide generation, resulting in high-precision control over diverse facial aspects without additional training.
Recent advances in diffusion models have significantly improved text-to-face generation, but achieving fine-grained control over facial features remains a challenge. Existing methods often require training additional modules to handle specific controls such as identity, attributes, or age, making them inflexible and resource-intensive. We propose ExpertGen, a training-free framework that leverages pre-trained expert models such as face recognition, facial attribute recognition, and age estimation networks to guide generation with fine control. Our approach uses a latent consistency model to ensure realistic and in-distribution predictions at each diffusion step, enabling accurate guidance signals to effectively steer the diffusion process. We show qualitatively and quantitatively that expert models can guide the generation process with high precision, and multiple experts can collaborate to enable simultaneous control over diverse facial aspects. By allowing direct integration of off-the-shelf expert models, our method transforms any such model into a plug-and-play component for controllable face generation.