AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head Reenactment
This addresses the problem of realistic animation head reenactment for applications like the metaverse, though it is incremental as it builds on existing reenactment models with a new dataset and method.
The authors introduced AnimeCeleb, a large-scale dataset for animation head reenactment using 3D models to generate images with detailed pose annotations, enabling high-quality results not achievable with existing datasets. They also proposed a novel pose mapping method for cross-domain head reenactment, showing superiority over state-of-the-art methods in experiments.
We present a novel Animation CelebHeads dataset (AnimeCeleb) to address an animation head reenactment. Different from previous animation head datasets, we utilize 3D animation models as the controllable image samplers, which can provide a large amount of head images with their corresponding detailed pose annotations. To facilitate a data creation process, we build a semi-automatic pipeline leveraging an open 3D computer graphics software with a developed annotation system. After training with the AnimeCeleb, recent head reenactment models produce high-quality animation head reenactment results, which are not achievable with existing datasets. Furthermore, motivated by metaverse application, we propose a novel pose mapping method and architecture to tackle a cross-domain head reenactment task. During inference, a user can easily transfer one's motion to an arbitrary animation head. Experiments demonstrate the usefulness of the AnimeCeleb to train animation head reenactment models, and the superiority of our cross-domain head reenactment model compared to state-of-the-art methods. Our dataset and code are available at https://github.com/kangyeolk/AnimeCeleb.