Althea: Human-AI Collaboration for Fact-Checking and Critical Reasoning
This addresses the need for more effective human-AI collaboration in fact-checking for users navigating online information, though it is incremental as it builds on existing verification methods with structured reasoning enhancements.
The paper tackled the problem of scalable and trustworthy fact-checking by introducing Althea, a retrieval-augmented system that supports user-driven evaluation of online claims, achieving a Macro-F1 of 0.44 on the AVeriTeC benchmark and showing that guided interaction improves accuracy and confidence in user studies.
The web's information ecosystem demands fact-checking systems that are both scalable and epistemically trustworthy. Automated approaches offer efficiency but often lack transparency, while human verification remains slow and inconsistent. We introduce Althea, a retrieval-augmented system that integrates question generation, evidence retrieval, and structured reasoning to support user-driven evaluation of online claims. On the AVeriTeC benchmark, Althea achieves a Macro-F1 of 0.44, outperforming standard verification pipelines and improving discrimination between supported and refuted claims. We further evaluate Althea through a controlled user study and a longitudinal survey experiment (N = 642), comparing three interaction modes that vary in the degree of scaffolding: an Exploratory mode with guided reasoning, a Summary mode providing synthesized verdicts, and a Self-search mode that offers procedural guidance without algorithmic intervention. Results show that guided interaction produces the strongest immediate gains in accuracy and confidence, while self-directed search yields the most persistent improvements over time. This pattern suggests that performance gains are not driven solely by effort or exposure, but by how cognitive work is structured and internalized.