How Reasoning Shapes Social Bias in LLM-Generated Code?
This work is significant for developers and users of LLMs for code generation, as it identifies and mitigates social bias that can arise during the reasoning process, which is an incremental improvement over existing bias mitigation strategies.
This paper investigates social bias in reasoning-based code generation by LLMs, finding that reasoning generally reduces bias from an average rate of 0.64 to 0.40, but often at the cost of code quality, which drops from 0.72 to 0.59. To address this, they propose ProbeDebias, a framework that detects and rewrites biased reasoning, achieving 87.76% F1 for detection and reducing code bias by 83.73% while improving quality by 9.79%-36.79% compared to SOTA baselines.
Large language models (LLMs) are increasingly used for code generation, yet generated programs may exhibit social bias through unfair or differential treatment of sensitive demographic attributes. While prior work mainly studies direct code generation, bias in reasoning-based generation remains underexplored. We conduct the first systematic study of social bias in reasoning-based code generation, evaluating 9 standard LLMs and large reasoning models (LRMs) on realistic bias-sensitive tasks across three human-centered decision scenarios. We find that reasoning generally reduces bias, lowering the average bias rate from 0.64 to 0.40, but the effect varies substantially across models. Meanwhile, code quality is not consistently preserved, with the average quality dropping from 0.72 to 0.59. Biased reasoning strongly predicts biased code, and adjusting generation configurations alone is insufficient for robust mitigation. Based on these findings, we propose ProbeDebias, a reasoning-aware framework that detects and rewrites biased reasoning traces before code generation. ProbeDebias achieves 87.76% F1 for reasoning-bias detection and reduces code bias by 83.73% on average while largely preserving quality. Compared with SOTA baselines, it further reduces average bias by 52.70%-54.42% and improves quality by 9.79%-36.79%. These results highlight the value of reasoning-stage analysis for trustworthy code generation.