Automated Fact-Checking in Dialogue: Are Specialized Models Needed?
This addresses the challenge of building practical fact-checking systems for conversational AI, though it is incremental as it adapts existing methods rather than introducing a new paradigm.
The paper tackles the problem of fact-checking models performing poorly on dialogue claims, showing that fine-tuning for dialogue hurts performance on stand-alone claims. It presents techniques like retrieval adaptation and input transformation to enable a single model to handle both dialogue and stand-alone claims competitively, maintaining accuracy on stand-alone claims while matching state-of-the-art dialogue-specific models.
Prior research has shown that typical fact-checking models for stand-alone claims struggle with claims made in dialogues. As a solution, fine-tuning these models on labelled dialogue data has been proposed. However, creating separate models for each use case is impractical, and we show that fine-tuning models for dialogue results in poor performance on typical fact-checking. To overcome this challenge, we present techniques that allow us to use the same models for both dialogue and typical fact-checking. These mainly focus on retrieval adaptation and transforming conversational inputs so that they can be accurately predicted by models trained on stand-alone claims. We demonstrate that a typical fact-checking model incorporating these techniques is competitive with state-of-the-art models fine-tuned for dialogue, while maintaining its accuracy on stand-alone claims.