Using multimodal speech production data to evaluate articulatory animation for audiovisual speech synthesis
This work tackles the problem of enhancing realism in audiovisual speech synthesis for applications like virtual assistants or entertainment, but it appears incremental as it builds on existing data-driven approaches without introducing a new paradigm.
The paper addresses the low quality of intraoral articulator animation in audiovisual speech synthesis by proposing the use of multimodal speech production data, which could significantly improve animation quality compared to current simple rules or viseme morphing methods.
The importance of modeling speech articulation for high-quality audiovisual (AV) speech synthesis is widely acknowledged. Nevertheless, while state-of-the-art, data-driven approaches to facial animation can make use of sophisticated motion capture techniques, the animation of the intraoral articulators (viz. the tongue, jaw, and velum) typically makes use of simple rules or viseme morphing, in stark contrast to the otherwise high quality of facial modeling. Using appropriate speech production data could significantly improve the quality of articulatory animation for AV synthesis.