Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation
This addresses safety risks for high-stakes societal applications like public policy and healthcare, though it appears incremental as a proof-of-concept framework.
The paper tackles the problem of ensuring language model safety by projecting how model-generated advice could propagate through societal systems over time, achieving over 20% improvement on a new dataset of indirect harm scenarios and an average win rate exceeding 70% against baselines on existing safety benchmarks.
Given the growing influence of language model-based agents on high-stakes societal decisions, from public policy to healthcare, ensuring their beneficial impact requires understanding the far-reaching implications of their suggestions. We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems on a macroscopic scale over time, enabling more robust alignment. To assess the long-term safety awareness of language models, we also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts. Our approach achieves not only over 20% improvement on the new dataset but also an average win rate exceeding 70% against strong baselines on existing safety benchmarks (AdvBench, SafeRLHF, WildGuardMix), suggesting a promising direction for safer agents.