Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
This work addresses the problem of enabling AI systems to adhere to unspoken social norms, which is crucial for their integration into human environments.
This paper explores whether natural language generation (NLG) models can act as behavioral priors for AI systems in social environments by generating morally constrained actions, anticipating consequences, and explaining norms. It introduces 'Moral Stories', a new crowd-sourced dataset of structured narratives for social reasoning, and proposes decoding strategies that significantly improve the quality of generated outputs compared to strong baselines.
In social settings, much of human behavior is governed by unspoken rules of conduct. For artificial systems to be fully integrated into social environments, adherence to such norms is a central prerequisite. We investigate whether contemporary NLG models can function as behavioral priors for systems deployed in social settings by generating action hypotheses that achieve predefined goals under moral constraints. Moreover, we examine if models can anticipate likely consequences of (im)moral actions, or explain why certain actions are preferable by generating relevant norms. For this purpose, we introduce 'Moral Stories', a crowd-sourced dataset of structured, branching narratives for the study of grounded, goal-oriented social reasoning. Finally, we propose decoding strategies that effectively combine multiple expert models to significantly improve the quality of generated actions, consequences, and norms compared to strong baselines, e.g. though abductive reasoning.