Can We Locate and Prevent Stereotypes in LLMs?
This addresses the issue of harmful societal biases perpetuated by LLMs, which is a domain-specific concern for AI ethics and fairness, but the work appears incremental as it builds on existing bias detection efforts.
The study tackled the problem of locating stereotype-related activations within large language models like GPT-2 Small and Llama 3.2, aiming to map 'bias fingerprints' for mitigation, but no concrete results or numbers were reported.
Stereotypes in large language models (LLMs) can perpetuate harmful societal biases. Despite the widespread use of models, little is known about where these biases reside in the neural network. This study investigates the internal mechanisms of GPT 2 Small and Llama 3.2 to locate stereotype related activations. We explore two approaches: identifying individual contrastive neuron activations that encode stereotypes, and detecting attention heads that contribute heavily to biased outputs. Our experiments aim to map these "bias fingerprints" and provide initial insights for mitigating stereotypes.