Leveraging Visemes for Better Visual Speech Representation and Lip Reading
This work addresses the challenge of improving lip reading accuracy for applications in speech recognition and human-computer interaction, representing a strong specific gain in the domain.
The paper tackled the problem of low accuracy in lip reading systems by leveraging visemes to extract more discriminative video features, resulting in a 9.1% relative reduction in word error rate compared to state-of-the-art methods.
Lip reading is a challenging task that has many potential applications in speech recognition, human-computer interaction, and security systems. However, existing lip reading systems often suffer from low accuracy due to the limitations of video features. In this paper, we propose a novel approach that leverages visemes, which are groups of phonetically similar lip shapes, to extract more discriminative and robust video features for lip reading. We evaluate our approach on various tasks, including word-level and sentence-level lip reading, and audiovisual speech recognition using the Arman-AV dataset, a largescale Persian corpus. Our experimental results show that our viseme based approach consistently outperforms the state-of-theart methods in all these tasks. The proposed method reduces the lip-reading word error rate (WER) by 9.1% relative to the best previous method.