On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
This work addresses safety problems for real-world deployment of conversational AI, though it is incremental as it builds on prior safety research with a focus on context-sensitive aspects.
The authors tackled the problem of dialogue safety in conversational models by proposing a taxonomy for context-sensitive unsafety and compiling the DiaSafety dataset, showing that existing safety tools fail severely on it, and they trained a classifier that revealed concerning safety issues in popular models.
Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corresponding dataset is scarce. We propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings, with focuses on context-sensitive unsafety, which is under-explored in prior works. To spur research in this direction, we compile DiaSafety, a dataset with rich context-sensitive unsafe examples. Experiments show that existing safety guarding tools fail severely on our dataset. As a remedy, we train a dialogue safety classifier to provide a strong baseline for context-sensitive dialogue unsafety detection. With our classifier, we perform safety evaluations on popular conversational models and show that existing dialogue systems still exhibit concerning context-sensitive safety problems.