Biases in the Blind Spot: Detecting What LLMs Fail to Mention
This work addresses the issue of hidden biases in LLMs for researchers and practitioners, offering a scalable method for automatic bias discovery, though it is incremental as it builds on existing bias evaluation concepts.
The authors tackled the problem of detecting unverbalized biases in Large Language Models (LLMs) that are not mentioned in chain-of-thought reasoning, and their automated pipeline successfully discovered previously unknown biases like Spanish fluency and English proficiency across six LLMs on three decision tasks.
Large Language Models (LLMs) often provide chain-of-thought (CoT) reasoning traces that appear plausible, but may hide internal biases. We call these *unverbalized biases*. Monitoring models via their stated reasoning is therefore unreliable, and existing bias evaluations typically require predefined categories and hand-crafted datasets. In this work, we introduce a fully automated, black-box pipeline for detecting task-specific unverbalized biases. Given a task dataset, the pipeline uses LLM autoraters to generate candidate bias concepts. It then tests each concept on progressively larger input samples by generating positive and negative variations, and applies statistical techniques for multiple testing and early stopping. A concept is flagged as an unverbalized bias if it yields statistically significant performance differences while not being cited as justification in the model's CoTs. We evaluate our pipeline across six LLMs on three decision tasks (hiring, loan approval, and university admissions). Our technique automatically discovers previously unknown biases in these models (e.g., Spanish fluency, English proficiency, writing formality). In the same run, the pipeline also validates biases that were manually identified by prior work (gender, race, religion, ethnicity). More broadly, our proposed approach provides a practical, scalable path to automatic task-specific bias discovery.