Framing Political Bias in Multilingual LLMs Across Pakistani Languages
This addresses a critical blind spot in bias auditing for low-resource, multilingual regions like Pakistan, where linguistic identity ties to political ideologies, though it is incremental as it applies existing methods to new data.
The study evaluated political bias in 13 state-of-the-art LLMs across five Pakistani languages, finding that while models generally reflect liberal-left orientations, they show more authoritarian framing in regional languages, indicating language-conditioned ideological modulation.
Large Language Models (LLMs) increasingly shape public discourse, yet most evaluations of political and economic bias have focused on high-resource, Western languages and contexts. This leaves critical blind spots in low-resource, multilingual regions such as Pakistan, where linguistic identity is closely tied to political, religious, and regional ideologies. We present a systematic evaluation of political bias in 13 state-of-the-art LLMs across five Pakistani languages: Urdu, Punjabi, Sindhi, Pashto, and Balochi. Our framework integrates a culturally adapted Political Compass Test (PCT) with multi-level framing analysis, capturing both ideological stance (economic/social axes) and stylistic framing (content, tone, emphasis). Prompts are aligned with 11 socio-political themes specific to the Pakistani context. Results show that while LLMs predominantly reflect liberal-left orientations consistent with Western training data, they exhibit more authoritarian framing in regional languages, highlighting language-conditioned ideological modulation. We also identify consistent model-specific bias patterns across languages. These findings show the need for culturally grounded, multilingual bias auditing frameworks in global NLP.