Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning
This work addresses cross-lingual phonetic representation learning, which is incremental as it builds on existing language modeling techniques by incorporating multilingual data and typological information.
The authors tackled the problem of modeling phone sequences across multiple languages by introducing polyglot language models that use shared symbol representations and typological conditioning, resulting in better generalization to held-out data and higher-quality phonetic feature representations compared to monolingual models.
We introduce polyglot language models, recurrent neural network models trained to predict symbol sequences in many different languages using shared representations of symbols and conditioning on typological information about the language to be predicted. We apply these to the problem of modeling phone sequences---a domain in which universal symbol inventories and cross-linguistically shared feature representations are a natural fit. Intrinsic evaluation on held-out perplexity, qualitative analysis of the learned representations, and extrinsic evaluation in two downstream applications that make use of phonetic features show (i) that polyglot models better generalize to held-out data than comparable monolingual models and (ii) that polyglot phonetic feature representations are of higher quality than those learned monolingually.