Automatically Exposing Problems with Neural Dialog Models
This addresses the challenge of systematically identifying critical flaws in dialog models for developers and users, though it is incremental as it builds on existing manual and crowd-sourced approaches.
The paper tackles the problem of neural dialog models generating unsafe and inconsistent responses by proposing two methods, including reinforcement learning, to automatically trigger these issues, showing effectiveness in exposing safety and contradiction problems in state-of-the-art models.
Neural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by model designers through interactions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. In this paper, we propose two methods including reinforcement learning to automatically trigger a dialog model into generating problematic responses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models.