News Source Citing Patterns in AI Search Systems
This research addresses the problem of understanding citation biases in AI search systems for users and developers, though it is incremental as it builds on existing concerns about information gatekeeping.
The study analyzed citation patterns in AI search systems using data from the AI Search Arena, finding that 9% of over 366,000 citations reference news sources, with a concentration on a few outlets and a liberal bias, but user satisfaction was not significantly affected by the political leaning or quality of these sources.
AI-powered search systems are emerging as new information gatekeepers, fundamentally transforming how users access news and information. Despite their growing influence, the citation patterns of these systems remain poorly understood. We address this gap by analyzing data from the AI Search Arena, a head-to-head evaluation platform for AI search systems. The dataset comprises over 24,000 conversations and 65,000 responses from models across three major providers: OpenAI, Perplexity, and Google. Among the over 366,000 citations embedded in these responses, 9% reference news sources. We find that while models from different providers cite distinct news sources, they exhibit shared patterns in citation behavior. News citations concentrate heavily among a small number of outlets and display a pronounced liberal bias, though low-credibility sources are rarely cited. User preference analysis reveals that neither the political leaning nor the quality of cited news sources significantly influences user satisfaction. These findings reveal significant challenges in current AI search systems and have important implications for their design and governance.