Creating Large Language Model Resistant Exams: Guidelines and Strategies
This addresses the challenge for educators in adapting assessments to modern AI tools, though it is incremental as it builds on existing concerns about academic integrity.
The paper tackles the problem of maintaining academic integrity in the face of Large Language Models like ChatGPT by proposing guidelines for designing LLM-resistant exams, such as using real-world scenarios and non-textual information, without providing specific performance numbers.
The proliferation of Large Language Models (LLMs), such as ChatGPT, has raised concerns about their potential impact on academic integrity, prompting the need for LLM-resistant exam designs. This article investigates the performance of LLMs on exams and their implications for assessment, focusing on ChatGPT's abilities and limitations. We propose guidelines for creating LLM-resistant exams, including content moderation, deliberate inaccuracies, real-world scenarios beyond the model's knowledge base, effective distractor options, evaluating soft skills, and incorporating non-textual information. The article also highlights the significance of adapting assessments to modern tools and promoting essential skills development in students. By adopting these strategies, educators can maintain academic integrity while ensuring that assessments accurately reflect contemporary professional settings and address the challenges and opportunities posed by artificial intelligence in education.