Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European Countries
This work addresses potential biases in multilingual AI models for European languages, highlighting risks of discrimination in applications like hiring or social analysis, though it is incremental as it builds on existing embedding analysis methods.
The study analyzed how multilingual sentence embeddings represent European countries and occupations across 12 languages, finding that the primary dimensions are geopolitical (Eastern vs. Western Europe) and economic (GDP), with job prestige as a distinct dimension largely uncorrelated with country features, except in one small model where a link suggests potential nationality-based discrimination.
We study how multilingual sentence representations capture European countries and occupations and how this differs across European languages. We prompt the models with templated sentences that we machine-translate into 12 European languages and analyze the most prominent dimensions in the embeddings.Our analysis reveals that the most prominent feature in the embedding is the geopolitical distinction between Eastern and Western Europe and the country's economic strength in terms of GDP. When prompted specifically for job prestige, the embedding space clearly distinguishes high and low-prestige jobs. The occupational dimension is uncorrelated with the most dominant country dimensions in three out of four studied models. The exception is a small distilled model that exhibits a connection between occupational prestige and country of origin, which is a potential source of nationality-based discrimination. Our findings are consistent across languages.