Alberto Piatti

Francesca Mangili, Giorgia Adorni, Alberto Piatti et al.

Automatic assessment of learner competencies is a fundamental task in intelligent tutoring systems. An assessment rubric typically and effectively describes relevant competencies and competence levels. This paper presents an approach to deriving a learner model directly from an assessment rubric defining some (partial) ordering of competence levels. The model is based on Bayesian networks and exploits logical gates with uncertainty (often referred to as noisy gates) to reduce the number of parameters of the model, so to simplify their elicitation by experts and allow real-time inference in intelligent tutoring systems. We illustrate how the approach can be applied to automatize the human assessment of an activity developed for testing computational thinking skills. The simple elicitation of the model starting from the assessment rubric opens up the possibility of quickly automating the assessment of several tasks, making them more easily exploitable in the context of adaptive assessment tools and intelligent tutoring systems.

Giorgia Adorni, Francesca Mangili, Alberto Piatti et al.

In modern and personalised education, there is a growing interest in developing learners' competencies and accurately assessing them. In a previous work, we proposed a procedure for deriving a learner model for automatic skill assessment from a task-specific competence rubric, thus simplifying the implementation of automated assessment tools. The previous approach, however, suffered two main limitations: (i) the ordering between competencies defined by the assessment rubric was only indirectly modelled; (ii) supplementary skills, not under assessment but necessary for accomplishing the task, were not included in the model. In this work, we address issue (i) by introducing dummy observed nodes, strictly enforcing the skills ordering without changing the network's structure. In contrast, for point (ii), we design a network with two layers of gates, one performing disjunctive operations by noisy-OR gates and the other conjunctive operations through logical ANDs. Such changes improve the model outcomes' coherence and the modelling tool's flexibility without compromising the model's compact parametrisation, interpretability and simple experts' elicitation. We used this approach to develop a learner model for Computational Thinking (CT) skills assessment. The CT-cube skills assessment framework and the Cross Array Task (CAT) are used to exemplify it and demonstrate its feasibility.

Giorgia Adorni, Alberto Piatti

Algorithmic thinking (AT) is a critical skill in today's digital society, and it is indispensable not only in computer science-related fields but also in everyday problem-solving. As a foundational component of digital education and literacy, fostering AT skills is increasingly relevant for all students and should become a standard part of compulsory education. However, successfully integrating AT into formal education requires effective teaching strategies and robust and scalable assessment procedures. In this paper, we present the design and development process of the virtual Cross Array Task (CAT), a digital adaptation of an unplugged assessment activity aimed at evaluating algorithmic skills in Swiss compulsory education. The development process followed iterative design cycles, incorporating expert evaluations to refine the tool's usability, accessibility and functionality. A participatory design study played a dual role in shaping the platform. First, it gathered valuable insights from end users, including students and teachers, to ensure the tool's relevance and practicality in classroom settings. Second, it facilitated the collection and preliminary analysis of data related to students' AT skills, providing an initial evaluation of the tool's assessment capabilities across various developmental stages. This was achieved through a pilot study involving a diverse group of students aged 4 to 12, spanning preschool to lower secondary school levels. The resulting instrument features multilingual support and includes both gesture-based and visual block-based programming interfaces, making it accessible to a broad range of learners. Findings from the pilot study demonstrate the platform's usability and accessibility, as well as its suitability for assessing AT skills, with preliminary results showing its ability to cater to diverse age groups and educational contexts.