MegaPortraits: One-shot Megapixel Neural Head Avatars
This work addresses the challenge of generating realistic, high-resolution avatars for applications like virtual reality or video conferencing, representing an incremental advance in neural avatar technology.
The paper tackles the problem of creating high-resolution neural head avatars from a single image, specifically in cross-driving scenarios where appearance differs significantly, and achieves megapixel resolution with improved quality and generalization over competitors.
In this work, we advance the neural head avatar technology to the megapixel resolution while focusing on the particularly challenging task of cross-driving synthesis, i.e., when the appearance of the driving image is substantially different from the animated source image. We propose a set of new neural architectures and training methods that can leverage both medium-resolution video data and high-resolution image data to achieve the desired levels of rendered image quality and generalization to novel views and motion. We demonstrate that suggested architectures and methods produce convincing high-resolution neural avatars, outperforming the competitors in the cross-driving scenario. Lastly, we show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time and locks the identities of neural avatars to several dozens of pre-defined source images. Real-time operation and identity lock are essential for many practical applications head avatar systems.