Rethinking AI Evaluation in Education: The TEACH-AI Framework and Benchmark for Generative AI Assistants
This work addresses the need for more holistic AI evaluation in education for designers, developers, researchers, and policymakers, though it is incremental as it builds on existing literature.
The paper tackles the problem that existing AI evaluations in education focus too much on technical metrics and neglect human and ethical aspects, resulting in the TEACH-AI framework and toolkit for assessing generative AI systems in educational contexts.
As generative artificial intelligence (AI) continues to transform education, most existing AI evaluations rely primarily on technical performance metrics such as accuracy or task efficiency while overlooking human identity, learner agency, contextual learning processes, and ethical considerations. In this paper, we present TEACH-AI (Trustworthy and Effective AI Classroom Heuristics), a domain-independent, pedagogically grounded, and stakeholder-aligned framework with measurable indicators and a practical toolkit for guiding the design, development, and evaluation of generative AI systems in educational contexts. Built on an extensive literature review and synthesis, the ten-component assessment framework and toolkit checklist provide a foundation for scalable, value-aligned AI evaluation in education. TEACH-AI rethinks "evaluation" through sociotechnical, educational, theoretical, and applied lenses, engaging designers, developers, researchers, and policymakers across AI and education. Our work invites the community to reconsider what constructs "effective" AI in education and to design model evaluation approaches that promote co-creation, inclusivity, and long-term human, social, and educational impact.