TRIED: Truly Innovative and Effective AI Detection Benchmark, developed by WITNESS
This work tackles the challenge of deceptive synthetic media for global stakeholders, but it is incremental as it proposes a benchmark rather than a new detection method.
The report addresses the problem of AI detection tools underperforming in real-world scenarios by introducing the TRIED Benchmark, a framework for evaluating these tools based on real-world impact and innovation, aiming to improve accountability and transparency.
The proliferation of generative AI and deceptive synthetic media threatens the global information ecosystem, especially across the Global Majority. This report from WITNESS highlights the limitations of current AI detection tools, which often underperform in real-world scenarios due to challenges related to explainability, fairness, accessibility, and contextual relevance. In response, WITNESS introduces the Truly Innovative and Effective AI Detection (TRIED) Benchmark, a new framework for evaluating detection tools based on their real-world impact and capacity for innovation. Drawing on frontline experiences, deceptive AI cases, and global consultations, the report outlines how detection tools must evolve to become truly innovative and relevant by meeting diverse linguistic, cultural, and technological contexts. It offers practical guidance for developers, policy actors, and standards bodies to design accountable, transparent, and user-centered detection solutions, and incorporate sociotechnical considerations into future AI standards, procedures and evaluation frameworks. By adopting the TRIED Benchmark, stakeholders can drive innovation, safeguard public trust, strengthen AI literacy, and contribute to a more resilient global information credibility.