Yin Ping Cao

Fang Kang, Yin Cao, Haoyu Chen

Recent studies in speech-driven talking face generation achieve promising results, but their reliance on fixed-driven speech limits further applications (e.g., face-voice mismatch). Thus, we extend the task to a more challenging setting: given a face image and text to speak, generating both talking face animation and its corresponding speeches. Accordingly, we propose a novel framework, Face2VoiceSync, with several novel contributions: 1) Voice-Face Alignment, ensuring generated voices match facial appearance; 2) Diversity \& Manipulation, enabling generated voice control over paralinguistic features space; 3) Efficient Training, using a lightweight VAE to bridge visual and audio large-pretrained models, with significantly fewer trainable parameters than existing methods; 4) New Evaluation Metric, fairly assessing the diversity and identity consistency. Experiments show Face2VoiceSync achieves both visual and audio state-of-the-art performances on a single 40GB GPU.

Yin Cao, Bao Wang, Kelin Xia et al.

The matched interface and boundary (MIB) method has a proven ability for delivering the second order accuracy in handling elliptic interface problems with arbitrarily complex interface geometries. However, its collocation formulation requires relatively high solution regularity. Finite volume method (FVM) has its merit in dealing with conservation law problems and its integral formulation works well with relatively low solution regularity. We propose an MIB-FVM to take the advantages of both MIB and FVM for solving elliptic interface problems. We construct the proposed method on Cartesian meshes with vertex-centered control volumes. A large number of numerical experiments are designed to validate the present method in both two dimensional (2D) and three dimensional (3D) domains. It is found that the proposed MIB-FVM achieves the second order convergence for elliptic interface problems with complex interface geometries in both $L_{\infty}$ and $L_2$ norms.