Motion Manipulation via Unsupervised Keypoint Positioning in Face Animation
This work provides improved control over facial animation for virtual portrait generation, offering a new way to manipulate face motion for artists and developers in the animation domain.
The paper addresses the challenge of controllable face generation in unsupervised keypoint-based face animation, where existing methods struggle to decouple identity and motion. The proposed MMFA method introduces self-supervised representation learning to disentangle expressions from other motion information and a novel keypoint computation approach for arbitrary motion control. It also uses a variational autoencoder to enable unsupervised interpolation of facial expressions, demonstrating pronounced advantages over prior art in creating realistic animation and manipulating face motion.
Face animation deals with controlling and generating facial features with a wide range of applications. The methods based on unsupervised keypoint positioning can produce realistic and detailed virtual portraits. However, they cannot achieve controllable face generation since the existing keypoint decomposition pipelines fail to fully decouple identity semantics and intertwined motion information (e.g., rotation, translation, and expression). To address these issues, we present a new method, Motion Manipulation via unsupervised keypoint positioning in Face Animation (MMFA). We first introduce self-supervised representation learning to encode and decode expressions in the latent feature space and decouple them from other motion information. Secondly, we propose a new way to compute keypoints aiming to achieve arbitrary motion control. Moreover, we design a variational autoencoder to map expression features to a continuous Gaussian distribution, allowing us for the first time to interpolate facial expressions in an unsupervised framework. We have conducted extensive experiments on publicly available datasets to validate the effectiveness of MMFA, which show that MMFA offers pronounced advantages over prior arts in creating realistic animation and manipulating face motion.