Learn What NOT to Learn: Towards Generative Safety in Chatbots
This addresses safety issues in open-domain chatbots for users and developers, representing a novel method rather than an incremental improvement.
The paper tackles the problem of generative chatbots producing unsafe content by introducing the LOT framework, which uses a contrastive loss to steer generations away from unsafe language, resulting in up to a four-fold reduction in toxicity and four to six-fold improvements in engagingness and fluency compared to baselines.
Conversational models that are generative and open-domain are particularly susceptible to generating unsafe content since they are trained on web-based social data. Prior approaches to mitigating this issue have drawbacks, such as disrupting the flow of conversation, limited generalization to unseen toxic input contexts, and sacrificing the quality of the dialogue for the sake of safety. In this paper, we present a novel framework, named "LOT" (Learn NOT to), that employs a contrastive loss to enhance generalization by learning from both positive and negative training signals. Our approach differs from the standard contrastive learning framework in that it automatically obtains positive and negative signals from the safe and unsafe language distributions that have been learned beforehand. The LOT framework utilizes divergence to steer the generations away from the unsafe subspace and towards the safe subspace while sustaining the flow of conversation. Our approach is memory and time-efficient during decoding and effectively reduces toxicity while preserving engagingness and fluency. Empirical results indicate that LOT reduces toxicity by up to four-fold while achieving four to six-fold higher rates of engagingness and fluency compared to baseline models. Our findings are further corroborated by human evaluation.