Wikipedia in the Era of LLMs: Evolution and Risks
This work highlights emerging risks for Wikipedia users and NLP researchers from LLM-generated content, though it is incremental in documenting early-stage impacts.
The paper analyzes how Large Language Models (LLMs) are affecting Wikipedia, finding that LLMs have influenced about 1%-2% of articles in certain categories and could inflate machine translation benchmark scores and reduce retrieval-augmented generation effectiveness if knowledge bases become polluted.
In this paper, we present a thorough analysis of the impact of Large Language Models (LLMs) on Wikipedia, examining the evolution of Wikipedia through existing data and using simulations to explore potential risks. We begin by analyzing page views and article content to study Wikipedia's recent changes and assess the impact of LLMs. Subsequently, we evaluate how LLMs affect various Natural Language Processing (NLP) tasks related to Wikipedia, including machine translation and retrieval-augmented generation (RAG). Our findings and simulation results reveal that Wikipedia articles have been influenced by LLMs, with an impact of approximately 1%-2% in certain categories. If the machine translation benchmark based on Wikipedia is influenced by LLMs, the scores of the models may become inflated, and the comparative results among models might shift as well. Moreover, the effectiveness of RAG might decrease if the knowledge base becomes polluted by LLM-generated content. While LLMs have not yet fully changed Wikipedia's language and knowledge structures, we believe that our empirical findings signal the need for careful consideration of potential future risks.