Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture
This addresses a specific challenge in social media, animation, and entertainment by enabling flexible expression deformation for caricatures, but it is incremental as it extends existing 3DMM techniques to a new domain.
The paper tackles the problem of deforming artist-drawn caricatures to match normal face expressions while preserving identity exaggeration, achieving visually good results in shape and color spaces through a novel method combining geometric optimization and a cGAN.
The problem of deforming an artist-drawn caricature according to a given normal face expression is of interest in applications such as social media, animation and entertainment. This paper presents a solution to the problem, with an emphasis on enhancing the ability to create desired expressions and meanwhile preserve the identity exaggeration style of the caricature, which imposes challenges due to the complicated nature of caricatures. The key of our solution is a novel method to model caricature expression, which extends traditional 3DMM representation to caricature domain. The method consists of shape modelling and texture generation for caricatures. Geometric optimization is developed to create identity-preserving blendshapes for reconstructing accurate and stable geometric shape, and a conditional generative adversarial network (cGAN) is designed for generating dynamic textures under target expressions. The combination of both shape and texture components makes the non-trivial expressions of a caricature be effectively defined by the extension of the popular 3DMM representation and a caricature can thus be flexibly deformed into arbitrary expressions with good results visually in both shape and color spaces. The experiments demonstrate the effectiveness of the proposed method.