Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models
This addresses the problem of value alignment in LLMs for AI safety and ethics, revealing a specific conflation issue that could affect model behavior in applications like content generation or decision-making.
The study investigated whether large language models (LLMs) distinguish between moral, grammatical, and economic value, finding pervasive value entanglement where grammatical and economic valuation were overly influenced by moral value relative to human norms, and this conflation was repaired by selective ablation of morality-related activation vectors.
Value alignment of Large Language Models (LLMs) requires us to empirically measure these models' actual, acquired representation of value. Among the characteristics of value representation in humans is that they distinguish among value of different kinds. We investigate whether LLMs likewise distinguish three different kinds of good: moral, grammatical, and economic. By probing model behavior, embeddings, and residual stream activations, we report pervasive cases of value entanglement: a conflation between these distinct representations of value. Specifically, both grammatical and economic valuation was found to be overly influenced by moral value, relative to human norms. This conflation was repaired by selective ablation of the activation vectors associated with morality.