End-to-End Lip Reading in Romanian with Cross-Lingual Domain Adaptation and Lateral Inhibition
This work addresses the problem of limited lip reading models for underrepresented languages like Romanian, though it appears incremental as it builds on existing methods with domain adaptation.
The paper tackles lip reading for the underrepresented Romanian language by analyzing architectures and optimizations on the Wild LRRo dataset, achieving state-of-the-art results through cross-lingual domain adaptation and regularization methods.
Lip reading or visual speech recognition has gained significant attention in recent years, particularly because of hardware development and innovations in computer vision. While considerable progress has been obtained, most models have only been tested on a few large-scale datasets. This work addresses this shortcoming by analyzing several architectures and optimizations on the underrepresented, short-scale Romanian language dataset called Wild LRRo. Most notably, we compare different backend modules, demonstrating the effectiveness of adding ample regularization methods. We obtain state-of-the-art results using our proposed method, namely cross-lingual domain adaptation and unlabeled videos from English and German datasets to help the model learn language-invariant features. Lastly, we assess the performance of adding a layer inspired by the neural inhibition mechanism.