The mutual exclusivity bias of bilingual visually grounded speech models
This work addresses how bilingual language learning affects computational models, offering incremental insights into biases in AI language acquisition.
The study investigated mutual exclusivity bias in bilingual visually grounded speech models, finding that bilingual models generally show a weaker bias than monolingual ones, with analyses linking this to reduced variance in visual embeddings for familiar data.
Mutual exclusivity (ME) is a strategy where a novel word is associated with a novel object rather than a familiar one, facilitating language learning in children. Recent work has found an ME bias in a visually grounded speech (VGS) model trained on English speech with paired images. But ME has also been studied in bilingual children, who may employ it less due to cross-lingual ambiguity. We explore this pattern computationally using bilingual VGS models trained on combinations of English, French, and Dutch. We find that bilingual models generally exhibit a weaker ME bias than monolingual models, though exceptions exist. Analyses show that the combined visual embeddings of bilingual models have a smaller variance for familiar data, partly explaining the increase in confusion between novel and familiar concepts. We also provide new insights into why the ME bias exists in VGS models in the first place. Code and data: https://github.com/danoneata/me-vgs