Learning to love diligent trolls: Accounting for rater effects in the dialogue safety task
This addresses the challenge of improving chatbot safety through user feedback by making label inference more robust and cost-effective, though it is an incremental improvement over existing methods.
The paper tackles the problem of trolls providing incorrect labels in chatbot safety training data by proposing a latent class analysis (LCA) method inspired by automated essay scoring, which infers correct labels with high accuracy even when trolls are consistent and in the majority.
Chatbots have the risk of generating offensive utterances, which must be avoided. Post-deployment, one way for a chatbot to continuously improve is to source utterance/label pairs from feedback by live users. However, among users are trolls, who provide training examples with incorrect labels. To de-troll training data, previous work removed training examples that have high user-aggregated cross-validation (CV) error. However, CV is expensive; and in a coordinated attack, CV may be overwhelmed by trolls in number and in consistency among themselves. In the present work, I address both limitations by proposing a solution inspired by methodology in automated essay scoring (AES): have multiple users rate each utterance, then perform latent class analysis (LCA) to infer correct labels. As it does not require GPU computations, LCA is inexpensive. In experiments, I found that the AES-like solution can infer training labels with high accuracy when trolls are consistent, even when trolls are the majority.