Nicolas S. Holliman

Nicolas Steven Holliman

We present a case study in the use of machine+human mixed intelligence for visualization quality assessment, applying automated visualization quality metrics to support the human assessment of data visualizations produced as coursework by students taking higher education courses. A set of image informatics algorithms including edge congestion, visual saliency and colour analysis generate machine analysis of student visualizations. The insight from the image informatics outputs has proved helpful for the marker in assessing the work and is also provided to the students as part of a written report on their work. Student and external reviewer comments suggest that the addition of the image informatics outputs to the standard feedback document was a positive step. We review the ethical challenges of working with assessment data and of automating assessment processes.

Nicolas S. Holliman, Arzu Coltekin, Sara J. Fernstad et al.

Background: Even though data visualizations (and underlying data) almost always contain uncertainty, it remains complex to communicate and interpret uncertainty representations. Consequently, uncertainty visualizations for non-expert audiences are rare. Objective: our aim is to rigorously define and evaluate the novel use of visual entropy as a measure of shape that allows us to construct an ordered scale of glyphs for use in representing both uncertainty and value in 2D and 3D environments. Method: We use sample entropy as a numerical measure of visual entropy to construct a set of glyphs using R and Blender which vary in their complexity. Results: an exact binomial analysis of a pairwise comparison of the glyphs shows a majority of participants (n = 87) ordered each glyph as predicted by the visual entropy score with large effect size (Cohen's g > 0.25). We also evaluate whether the glyphs effectively represent uncertainty using a signal detection method in a search task. Participants (n = 15) were able to find glyphs representing uncertainty with high sensitivity and low error rates. Conclusion: visual entropy is a successful novel approach to representing ordered data and provides a channel that can allow the uncertainty of a measure to be presented alongside its mean value.