Predicting non-native speech perception using the Perceptual Assimilation Model and state-of-the-art acoustic models
This research addresses speech perception challenges for linguists and AI researchers, offering incremental insights by integrating traditional models with modern acoustic methods.
The study tackled how native language influences non-native speech perception by comparing the Perceptual Assimilation Model with fine-grained phonetic models using acoustic representations, finding that phoneme assimilation better predicts discrimination behavior and native language effects, with wav2vec 2.0 providing complementary low-level phonetic insights.
Our native language influences the way we perceive speech sounds, affecting our ability to discriminate non-native sounds. We compare two ideas about the influence of the native language on speech perception: the Perceptual Assimilation Model, which appeals to a mental classification of sounds into native phoneme categories, versus the idea that rich, fine-grained phonetic representations tuned to the statistics of the native language, are sufficient. We operationalize this idea using representations from two state-of-the-art speech models, a Dirichlet process Gaussian mixture model and the more recent wav2vec 2.0 model. We present a new, open dataset of French- and English-speaking participants' speech perception behaviour for 61 vowel sounds from six languages. We show that phoneme assimilation is a better predictor than fine-grained phonetic modelling, both for the discrimination behaviour as a whole, and for predicting differences in discriminability associated with differences in native language background. We also show that wav2vec 2.0, while not good at capturing the effects of native language on speech perception, is complementary to information about native phoneme assimilation, and provides a good model of low-level phonetic representations, supporting the idea that both categorical and fine-grained perception are used during speech perception.