Evaluating the robustness of source code plagiarism detection tools to pervasive plagiarism-hiding modifications
This work identifies a significant robustness gap in current source code plagiarism detection tools, which is a problem for educators trying to detect plagiarism in undergraduate computer science courses. This is an incremental finding.
This paper evaluates 11 source code plagiarism detection tools against pervasive plagiarism-hiding modifications commonly used by undergraduate students. The study found that existing tools are generally not robust to fine-grained structural transformations, though JPlag and Plaggie showed the greatest robustness among those tested.
Source code plagiarism is a common occurrence in undergraduate computer science education. In order to identify such cases, many source code plagiarism detection tools have been proposed. A source code plagiarism detection tool evaluates pairs of assignment submissions to detect indications of plagiarism. However, a plagiarising student will commonly apply plagiarism-hiding modifications to source code in an attempt to evade detection. Subsequently, prior work has implied that currently available source code plagiarism detection tools are not robust to the application of pervasive plagiarism-hiding modifications. In this article, 11 source code plagiarism detection tools are evaluated for robustness against plagiarism-hiding modifications. The tools are evaluated with data sets of simulated undergraduate plagiarism, constructed with source code modifications representative of undergraduate students. The results of the performed evaluations indicate that currently available source code plagiarism detection tools are not robust against modifications which apply fine-grained transformations to the source code structure. Of the evaluated tools, JPlag and Plaggie demonstrates the greatest robustness to different types of plagiarism-hiding modifications. However, the results also indicate that graph-based tools (specifically those that compare programs as program dependence graphs) show potentially greater robustness to pervasive plagiarism-hiding modifications.