Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool
This addresses the issue of AI misuse in academic publishing, which is crucial for maintaining research integrity, but it is incremental as it focuses on evaluating an existing tool rather than developing a new method.
The study tackled the problem of detecting AI-generated content in academic papers by evaluating the AI detection tool Originality.ai on a dataset of arXiv submissions from physics, mathematics, and computer science, achieving an accuracy rate of 98%.
Many people are interested in ChatGPT since it has become a prominent AIGC model that provides high-quality responses in various contexts, such as software development and maintenance. Misuse of ChatGPT might cause significant issues, particularly in public safety and education, despite its immense potential. The majority of researchers choose to publish their work on Arxiv. The effectiveness and originality of future work depend on the ability to detect AI components in such contributions. To address this need, this study will analyze a method that can see purposely manufactured content that academic organizations use to post on Arxiv. For this study, a dataset was created using physics, mathematics, and computer science articles. Using the newly built dataset, the following step is to put originality.ai through its paces. The statistical analysis shows that Originality.ai is very accurate, with a rate of 98%.