The Good, the Bad, and the Ugly: The Role of AI Quality Disclosure in Lie Detection
This addresses the issue of AI transparency and user trust in lie detection for applications like social media moderation, though it is incremental as it builds on existing research about AI disclosures and human-AI interaction.
The study tackled the problem of how low-quality AI advisors without quality disclosures can reduce people's ability to detect lies in text, finding that participants' truth-detection rates dropped below their own baseline when relying on such advisors, but recovered after disclosure, while high-quality advisors improved detection regardless of disclosure.
We investigate how low-quality AI advisors, lacking quality disclosures, can help spread text-based lies while seeming to help people detect lies. Participants in our experiment discern truth from lies by evaluating transcripts from a game show that mimicked deceptive social media exchanges on topics with objective truths. We find that when relying on low-quality advisors without disclosures, participants' truth-detection rates fall below their own abilities, which recovered once the AI's true effectiveness was revealed. Conversely, high-quality advisor enhances truth detection, regardless of disclosure. We discover that participants' expectations about AI capabilities contribute to their undue reliance on opaque, low-quality advisors.