Pierrick Bougault

William Guey, Wei Zhang, Pei-Luen Patrick Rau et al.

Background: Large language models (LLMs) harbor systematic biases that are particularly consequential in workplace and HR contexts, where their outputs increasingly influence hiring, job design, and organizational decisions. Existing bias-evaluation approaches remain methodologically fragmented, limiting practitioners' ability to assess deployment risks. Objective: This study introduces BiasLab, a multilingual dual-framing framework to quantify and compare directional output-level bias in LLMs, demonstrated across six workplace and HR-relevant topics. Methods: BiasLab combines mirrored affirmative and reverse prompt pairs, randomized wrapper perturbations, fixed-choice response constraints, and polarity-aligned scoring. Ten LLMs were evaluated across six topics (gender in leadership, employment gap candidates, age in hiring, remote versus office work, four-day versus five-day work weeks, and AI-assisted versus human-only hiring), spanning 12 languages and 30 iterations per framing direction, yielding 43,200 responses. Results: All ten models showed consistent directional preferences across every topic. A recurring asymmetric pattern emerged in which models rejected disfavored claims more strongly than they endorsed their opposites, a distinction invisible to single-frame designs. Conclusions: BiasLab provides a standardized, reproducible instrument for measuring directional preferences across models. Whether a preference constitutes bias in a fairness sense is topic-dependent: for protected attributes such as gender and age it maps onto equal-employment standards, whereas elsewhere it is better described as systematic preference. The framework lets organizations compare and vet models before adopting them for hiring.

William Guey, Pierrick Bougault, Wei Zhang et al.

Who invented the radio, Russia's Alexander Popov or Italy's Guglielmo Marconi? Was the telephone the achievement of Bell in the United States or Meucci in Italy? Does printing belong to China's Bi Sheng or Germany's Gutenberg? The answer depends not only on historical record but also on language and perspective. We analyse eleven widely used large language models across 21 disputed inventions and discoveries, evaluated in twelve languages and 75,896 responses. While models generally acknowledge that credit is contested, query language systematically affects which claimant is surfaced. Lower-status claimants are more likely to appear when questions are asked in their associated language, whereas dominant Anglophone figures remain stable across languages. These patterns persist after controlling for response length, model differences, historical prominence, and levels of national commemoration. Language thus acts as a switch that activates different national versions of the same history, producing systematically different national memories from the same question. We interpret this as evidence that large language models function as distributed systems of cultural memory, where language conditions which histories become visible, contributing to a computational form of banal nationalism.

William Guey, Pierrick Bougault

Large language models (LLMs) increasingly review and revise text, including their own. A documented self-preference bias (models favoring their own generations when acting as judges) raises the question of whether models also resist valid corrections to their own writing. We test this in a setting where "valid" is decided not by another model but by a deterministic verifier: instruction-following revision on IFEval. A model writes a draft; the official IFEval checker confirms the draft violates a constraint and that a candidate edit fixes it; the model then accepts or rejects that edit either as the genuine in-context author or as a fresh model that sees the draft neutrally. Across four mid-tier model families and 85 author-versus-fresh comparisons, we find no detectable self-preference: authors reject verified-good fixes to their own drafts at essentially the same rate as fresh models judging the same drafts (gap -5.1 pp, 95% CI [-12.9, +2.7]). A self-skepticism hint from a smaller pilot did not replicate at scale. The one robust observation is qualitative: when authors do reject a verified-good fix, 97% of their stated reasons are flaw-catching rather than preference, that is, about the character of rejections, not an elevated rate. Effects smaller than ~13 pp cannot be excluded at this sample size.