Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts
This work addresses fairness and bias in LLM persona generation for polarized geopolitical contexts, highlighting inconsistent model adjustments, but it is incremental as it builds on existing bias analysis without introducing new methods.
The study analyzed how five popular LLMs generate personas for Palestinian and Israeli identities across 640 conditions, finding that Palestinian profiles in war contexts were often linked to lower socioeconomic status and survival roles, while Israeli profiles maintained middle-class and professional attributes, with fairness prompts leading to inconsistent distributional changes.
Large language models (LLMs) are increasingly utilised for social simulation and persona generation, necessitating an understanding of how they represent geopolitical identities. In this paper, we analyse personas generated for Palestinian and Israeli identities by five popular LLMs across 640 experimental conditions, varying context (war vs non-war) and assigned roles. We observe significant distributional patterns in the generated attributes: Palestinian profiles in war contexts are frequently associated with lower socioeconomic status and survival-oriented roles, whereas Israeli profiles predominantly retain middle-class status and specialised professional attributes. When prompted with explicit instructions to avoid harmful assumptions, models exhibit diverse distributional changes, e.g., marked increases in non-binary gender inferences or a convergence toward generic occupational roles (e.g., "student"), while the underlying socioeconomic distinctions often remain. Furthermore, analysis of reasoning traces reveals an interesting dynamics between model reasoning and generation: while rationales consistently mention fairness-related concepts, the final generated personas follow the aforementioned diverse distributional changes. These findings illustrate a picture of how models interpret geopolitical contexts, while suggesting that they process fairness and adjust in varied ways; there is no consistent, direct translation of fairness concepts into representative outcomes.