LLM-Assisted Cheating Detection in Korean Language via Keystrokes
This addresses cheating detection in educational settings for Korean language, though it is incremental as it extends keystroke analysis to a new language and cognitive contexts.
The paper tackled detecting LLM-assisted cheating in Korean writing by analyzing keystrokes, using a dataset of 69 participants across tasks like paraphrasing and transcribing ChatGPT responses, and found that models outperformed humans, with temporal features effective in cognition-aware settings and rhythmic features generalizing better across cognitive scenarios.
This paper presents a keystroke-based framework for detecting LLM-assisted cheating in Korean, addressing key gaps in prior research regarding language coverage, cognitive context, and the granularity of LLM involvement. Our proposed dataset includes 69 participants who completed writing tasks under three conditions: Bona fide writing, paraphrasing ChatGPT responses, and transcribing ChatGPT responses. Each task spans six cognitive processes defined in Bloom's Taxonomy (remember, understand, apply, analyze, evaluate, and create). We extract interpretable temporal and rhythmic features and evaluate multiple classifiers under both Cognition-Aware and Cognition-Unaware settings. Temporal features perform well under Cognition-Aware evaluation scenarios, while rhythmic features generalize better under cross-cognition scenarios. Moreover, detecting bona fide and transcribed responses was easier than paraphrased ones for both the proposed models and human evaluators, with the models significantly outperforming the humans. Our findings affirm that keystroke dynamics facilitate reliable detection of LLM-assisted writing across varying cognitive demands and writing strategies, including paraphrasing and transcribing LLM-generated responses.