MyPortrait: Morphable Prior-Guided Personalized Portrait Generation
It addresses the problem of realistic portrait generation for computer vision applications, offering both real-time and high-quality versions, but appears incremental as it builds on existing morphable prior concepts.
The paper tackles generating high-quality dynamic talking faces with personalized details by proposing MyPortrait, a framework that incorporates personalized and morphable priors, achieving superior performance over state-of-the-art methods in various metrics.
Generating realistic talking faces is an interesting and long-standing topic in the field of computer vision. Although significant progress has been made, it is still challenging to generate high-quality dynamic faces with personalized details. This is mainly due to the inability of the general model to represent personalized details and the generalization problem to unseen controllable parameters. In this work, we propose Myportrait, a simple, general, and flexible framework for neural portrait generation. We incorporate personalized prior in a monocular video and morphable prior in 3D face morphable space for generating personalized details under novel controllable parameters. Our proposed framework supports both video-driven and audio-driven face animation given a monocular video of a single person. Distinguished by whether the test data is sent to training or not, our method provides a real-time online version and a high-quality offline version. Comprehensive experiments in various metrics demonstrate the superior performance of our method over the state-of-the-art methods. The code will be publicly available.