A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset
This provides a foundational resource for multi-modal speech tasks in Persian, addressing a gap for a low-resource language, though the dataset creation is incremental in nature.
The authors tackled the lack of large-scale audio-visual datasets for low-resource languages by creating the Arman-AV dataset, a 220-hour Persian corpus with 1,760 speakers, and proposed a viseme detection method that improved lip reading accuracy by 7% relative to previous approaches.
In recent years, significant progress has been made in automatic lip reading. But these methods require large-scale datasets that do not exist for many low-resource languages. In this paper, we have presented a new multipurpose audio-visual dataset for Persian. This dataset consists of almost 220 hours of videos with 1760 corresponding speakers. In addition to lip reading, the dataset is suitable for automatic speech recognition, audio-visual speech recognition, and speaker recognition. Also, it is the first large-scale lip reading dataset in Persian. A baseline method was provided for each mentioned task. In addition, we have proposed a technique to detect visemes (a visual equivalent of a phoneme) in Persian. The visemes obtained by this method increase the accuracy of the lip reading task by 7% relatively compared to the previously proposed visemes, which can be applied to other languages as well.