AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations
For researchers and practitioners in AI-mediated education, this dataset and fine-tuning approach enable privacy-preserving, locally deployable auditing of pedagogical risks in LLM-generated explanations.
This work introduces AIriskEval-edu-db2, a dataset of 1,639 K-12 educational explanations with risk annotations across five dimensions, and shows that supervised fine-tuning of a Llama 3.1 8B model on this dataset allows it to approach or outperform frontier models in pedagogical risk detection and explainability assessment.
This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K-12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks.