Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective
This work addresses bias in LLMs, a critical issue for fairness and ethics in AI, but it is incremental as it builds on existing interpretability methods without introducing a new paradigm.
The paper tackled the problem of social, demographic, and gender biases in LLMs like GPT-2 and Llama2 by using mechanistic interpretability to analyze their structural representation, finding that bias-related computations are highly localized in specific layers and that removing these components reduces biased outputs but also affects other NLP tasks.
Large Language Models (LLMs) are known to exhibit social, demographic, and gender biases, often as a consequence of the data on which they are trained. In this work, we adopt a mechanistic interpretability approach to analyze how such biases are structurally represented within models such as GPT-2 and Llama2. Focusing on demographic and gender biases, we explore different metrics to identify the internal edges responsible for biased behavior. We then assess the stability, localization, and generalizability of these components across dataset and linguistic variations. Through systematic ablations, we demonstrate that bias-related computations are highly localized, often concentrated in a small subset of layers. Moreover, the identified components change across fine-tuning settings, including those unrelated to bias. Finally, we show that removing these components not only reduces biased outputs but also affects other NLP tasks, such as named entity recognition and linguistic acceptability judgment because of the sharing of important components with these tasks.