Towards an Understanding and Explanation for Mixed-Initiative Artificial Scientific Text Detection
This addresses the challenge of plagiarism in academic contexts by improving detection accuracy and interpretability, though it is incremental as it builds on existing detection methods.
The paper tackled the problem of detecting machine-generated scientific text by identifying key differences between human and AI writing and proposing a mixed-initiative workflow that integrates human expertise with machine intelligence, demonstrating effectiveness through case studies and a user study with researchers.
Large language models (LLMs) have gained popularity in various fields for their exceptional capability of generating human-like text. Their potential misuse has raised social concerns about plagiarism in academic contexts. However, effective artificial scientific text detection is a non-trivial task due to several challenges, including 1) the lack of a clear understanding of the differences between machine-generated and human-written scientific text, 2) the poor generalization performance of existing methods caused by out-of-distribution issues, and 3) the limited support for human-machine collaboration with sufficient interpretability during the detection process. In this paper, we first identify the critical distinctions between machine-generated and human-written scientific text through a quantitative experiment. Then, we propose a mixed-initiative workflow that combines human experts' prior knowledge with machine intelligence, along with a visual analytics prototype to facilitate efficient and trustworthy scientific text detection. Finally, we demonstrate the effectiveness of our approach through two case studies and a controlled user study with proficient researchers. We also provide design implications for interactive artificial text detection tools in high-stakes decision-making scenarios.