Automatic Creativity Measurement in Scratch Programs Across Modalities
This work addresses the challenge of scalable creativity measurement in educational technologies, specifically for Scratch programming, though it is incremental as it adapts existing theoretical concepts to a practical domain.
The paper tackled the problem of measuring creativity in Scratch programming projects by developing a formal, efficiently computable measure based on fluency, flexibility, and originality, and trained a machine learning model on expert assessments. The results showed that the automatic assessment aligned more closely with human experts than the experts agreed among themselves, indicating its potential reliability.
Promoting creativity is considered an important goal of education, but creativity is notoriously hard to measure.In this paper, we make the journey fromdefining a formal measure of creativity that is efficientlycomputable to applying the measure in a practical domain. The measure is general and relies on coretheoretical concepts in creativity theory, namely fluency, flexibility, and originality, integratingwith prior cognitive science literature. We adapted the general measure for projects in the popular visual programming language Scratch.We designed a machine learning model for predicting the creativity of Scratch projects, trained and evaluated on human expert creativity assessments in an extensive user study. Our results show that opinions about creativity in Scratch varied widely across experts. The automatic creativity assessment aligned with the assessment of the human experts more than the experts agreed with each other. This is a first step in providing computational models for measuring creativity that can be applied to educational technologies, and to scale up the benefit of creativity education in schools.