Interpretable Unified Language Checking
This work addresses the need for interpretable and adaptive tools to detect misinformation, stereotypes, and hate speech in both human and machine-generated language, offering a unified approach that could benefit content moderation and safety applications.
The authors tackled the problem of detecting undesirable behaviors in language, such as non-factual, biased, and hateful content, by proposing a unified language checking method using LLMs, which achieved high performance and outperformed fully supervised baselines on tasks like fact-checking, stereotype detection, and hate speech detection with few-shot prompts.
Despite recent concerns about undesirable behaviors generated by large language models (LLMs), including non-factual, biased, and hateful language, we find LLMs are inherent multi-task language checkers based on their latent representations of natural and social knowledge. We present an interpretable, unified, language checking (UniLC) method for both human and machine-generated language that aims to check if language input is factual and fair. While fairness and fact-checking tasks have been handled separately with dedicated models, we find that LLMs can achieve high performance on a combination of fact-checking, stereotype detection, and hate speech detection tasks with a simple, few-shot, unified set of prompts. With the ``1/2-shot'' multi-task language checking method proposed in this work, the GPT3.5-turbo model outperforms fully supervised baselines on several language tasks. The simple approach and results suggest that based on strong latent knowledge representations, an LLM can be an adaptive and explainable tool for detecting misinformation, stereotypes, and hate speech.