Developing Story: Case Studies of Generative AI's Use in Journalism
This research highlights ethical concerns and the need for guidelines in AI use for journalism, addressing responsible practices for journalists and news agencies.
The study investigated how journalists use large language models (LLMs) by analyzing interactions from the WildChat dataset and matching them to published articles, finding that journalists often input sensitive materials like confidential correspondence to generate articles with limited editing (median ROUGE-L of 0.62).
Journalists are among the many users of large language models (LLMs). To better understand the journalist-AI interactions, we conduct a study of LLM usage by two news agencies through browsing the WildChat dataset, identifying candidate interactions, and verifying them by matching to online published articles. Our analysis uncovers instances where journalists provide sensitive material such as confidential correspondence with sources or articles from other agencies to the LLM as stimuli and prompt it to generate articles, and publish these machine-generated articles with limited intervention (median output-publication ROUGE-L of 0.62). Based on our findings, we call for further research into what constitutes responsible use of AI, and the establishment of clear guidelines and best practices on using LLMs in a journalistic context.