Building Interpretable Models for Moral Decision-Making
This work addresses the challenge of interpretability in AI for moral decision-making, which is important for researchers and developers in AI ethics, but it is incremental as it applies existing methods to a specific domain.
The researchers tackled the problem of understanding how neural networks make moral decisions by building a custom transformer model for trolley-style dilemmas, achieving 77% accuracy on Moral Machine data while enabling detailed interpretability analysis.
We build a custom transformer model to study how neural networks make moral decisions on trolley-style dilemmas. The model processes structured scenarios using embeddings that encode who is affected, how many people, and which outcome they belong to. Our 2-layer architecture achieves 77% accuracy on Moral Machine data while remaining small enough for detailed analysis. We use different interpretability techniques to uncover how moral reasoning distributes across the network, demonstrating that biases localize to distinct computational stages among other findings.