The Conversational Exam: A Scalable Assessment Design for the AI Era
For educators facing assessment validity challenges due to generative AI, this provides a practical, scalable solution that balances authentic practice with inherent validity.
The paper introduces the conversational exam, a scalable oral examination format where students code live and explain their reasoning, restoring assessment validity in the AI era. Testing with 58 students in small groups over two days, it demonstrates that oral exams can scale to typical class sizes.
Traditional assessment methods collapse when students use generative AI to complete work without genuine engagement, creating an illusion of competence where they believe they're learning but aren't. This paper presents the conversational exam -- a scalable oral examination format that restores assessment validity by having students code live while explaining their reasoning. Drawing on human-computer interaction principles, we examined 58 students in small groups across just two days, demonstrating that oral exams can scale to typical class sizes. The format combines authentic practice (students work with documentation and supervised AI access) with inherent validity (real-time performance cannot be faked). We provide detailed implementation guidance to help instructors adapt this approach, offering a practical path forward when many educators feel paralyzed between banning AI entirely or accepting that valid assessment is impossible.