Fighting AI with AI: AI-Agent Augmented DNS Blocking of LLM Services during Student Evaluations
This addresses academic integrity concerns for educators and institutions during student evaluations, though it is an incremental application of existing methods to a new domain.
The paper tackles the problem of students using LLM services to bypass critical thinking during proctored exams by introducing AI-Sinkhole, an AI-agent augmented DNS-based framework that dynamically blocks these services, achieving robust cross-lingual classification with an F1-score > 0.83.
The transformative potential of large language models (LLMs) in education, such as improving accessibility and personalized learning, is being eclipsed by significant challenges. These challenges stem from concerns that LLMs undermine academic assessment by enabling bypassing of critical thinking, leading to increased cognitive offloading. This emerging trend stresses the dual imperative of harnessing AI's educational benefits while safeguarding critical thinking and academic rigor in the evolving AI ecosystem. To this end, we introduce AI-Sinkhole, an AI-agent augmented DNS-based framework that dynamically discovers, semantically classifies, and temporarily network-wide blocks emerging LLM chatbot services during proctored exams. AI-Sinkhole offers explainable classification via quantized LLMs (LLama 3, DeepSeek-R1, Qwen-3) and dynamic DNS blocking with Pi-Hole. We also share our observations in using LLMs as explainable classifiers which achieved robust cross-lingual performance (F1-score > 0.83). To support future research and development in this domain initial codes with a readily deployable 'AI-Sinkhole' blockist is available on https://github.com/AIMLEdu/ai-sinkhole.