Alfie Abdul-Rahman

Yiwen Xing, Cristina Dondi, Rita Borgo et al.

The circulation of historical books has always been an area of interest for historians. However, the data used to represent the journey of a book across different places and times can be difficult for domain experts to digest due to buried geographical and chronological features within text-based presentations. This situation provides an opportunity for collaboration between visualization researchers and historians. This paper describes a design study where a variant of the Nine-Stage Framework was employed to develop a Visual Analytics (VA) tool called DanteExploreVis. This tool was designed to aid domain experts in exploring, explaining, and presenting book trade data from multiple perspectives. We discuss the design choices made and how each panel in the interface meets the domain requirements. We also present the results of a qualitative evaluation conducted with domain experts. The main contributions of this paper include: 1) the development of a VA tool to support domain experts in exploring, explaining, and presenting book trade data; 2) a comprehensive documentation of the iterative design, development, and evaluation process following the variant Nine-Stage Framework; 3) a summary of the insights gained and lessons learned from this design study in the context of the humanities field; and 4) reflections on how our approach could be applied in a more generalizable way.

Munkhtulga Battogtokh, Yiwen Xing, Cosmin Davidescu et al.

In natural language processing (NLP), text classification tasks are increasingly fine-grained, as datasets are fragmented into a larger number of classes that are more difficult to differentiate from one another. As a consequence, the semantic structures of datasets have become more complex, and model decisions more difficult to explain. Existing tools, suited for coarse-grained classification, falter under these additional challenges. In response to this gap, we worked closely with NLP domain experts in an iterative design-and-evaluation process to characterize and tackle the growing requirements in their workflow of developing fine-grained text classification models. The result of this collaboration is the development of SemLa, a novel visual analytics system tailored for 1) dissecting complex semantic structures in a dataset when it is spatialized in model embedding space, and 2) visualizing fine-grained nuances in the meaning of text samples to faithfully explain model reasoning. This paper details the iterative design study and the resulting innovations featured in SemLa. The final design allows contrastive analysis at different levels by unearthing lexical and conceptual patterns including biases and artifacts in data. Expert feedback on our final design and case studies confirm that SemLa is a useful tool for supporting model validation and debugging as well as data annotation.

Saiful Khan, Phong H. Nguyen, Alfie Abdul-Rahman et al.

In the process of developing an infrastructure for providing visualization and visual analytics (VIS) tools to epidemiologists and modeling scientists, we encountered a technical challenge for applying a number of visual designs to numerous datasets rapidly and reliably with limited development resources. In this paper, we present a technical solution to address this challenge. Operationally, we separate the tasks of data management, visual designs, and plots and dashboard deployment in order to streamline the development workflow. Technically, we utilize: an ontology to bring datasets, visual designs, and deployable plots and dashboards under the same management framework; multi-criteria search and ranking algorithms for discovering potential datasets that match a visual design; and a purposely-design user interface for propagating each visual design to appropriate datasets (often in tens and hundreds) and quality-assuring the propagation before the deployment. This technical solution has been used in the development of the RAMPVIS infrastructure for supporting a consortium of epidemiologists and modeling scientists through visualization.

Min Chen, Alfie Abdul-Rahman, Deborah Silver et al.

Many visual representations, such as volume-rendered images and metro maps, feature a noticeable amount of information loss. At a glance, there seem to be numerous opportunities for viewers to misinterpret the data being visualized, hence undermining the benefits of these visual representations. In practice, there is little doubt that these visual representations are useful. The recently-proposed information-theoretic measure for analyzing the cost-benefit ratio of visualization processes can explain such usefulness experienced in practice, and postulate that the viewers' knowledge can reduce the potential distortion (e.g., misinterpretation) due to information loss. This suggests that viewers' knowledge can be estimated by comparing the potential distortion without any knowledge and the actual distortion with some knowledge. In this paper, we describe several case studies for collecting instances that can (i) support the evaluation of several candidate measures for estimating the potential distortion distortion in visualization, and (ii) demonstrate their applicability in practical scenarios. Because the theoretical discourse on choosing an appropriate bounded measure for estimating the potential distortion is yet conclusive, it is the real world data about visualization further informs the selection of a bounded measure, providing practical evidence to aid a theoretical conclusion. Meanwhile, once we can measure the potential distortion in a bounded manner, we can interpret the numerical values characterizing the benefit of visualization more intuitively.

Alexandra Diehl, Matthias Kraus, Alfie Abdul-Rahman et al.

Visualization guidelines, if defined properly, are invaluable to both practical applications and the theoretical foundation of visualization. In this paper, we present a collection of research activities for studying visualization guidelines according to Grounded Theory (GT). We used the discourses at VisGuides, which is an online discussion forum for visualization guidelines, as the main data source for enabling data-driven research processes as advocated by the grounded theory methodology. We devised a categorization scheme focusing on observing how visualization guidelines were featured in different threads and posts at VisGuides, and coded all 248 posts between September 27, 2017 (when VisGuides was first launched) and March 13, 2019. To complement manual categorization and coding, we used text analysis and visualization to help reveal patterns that may have been missed by the manual effort and summary statistics. To facilitate theoretical sampling and negative case analysis, we made an in-depth analysis of the 148 posts (with both questions and replies) related to a student assignment of a visualization course. Inspired by two discussion threads at VisGuides, we conducted two controlled empirical studies to collect further data to validate specific visualization guidelines. Through these activities guided by grounded theory, we have obtained some new findings about visualization guidelines.

Min Chen, Alfie Abdul-Rahman, David H. Laidlaw

In each of the last five years, a few dozen empirical studies appeared in visualization journals and conferences. The existing empirical studies have already featured a large number of variables. There are many more variables yet to be studied. While empirical studies enable us to obtain knowledge and insight about visualization processes through observation and analysis of user experience, it seems to be a stupendous challenge for exploring such a huge variable space at the current pace. In this position paper, we discuss the implication of not being able to explore this space effectively and efficiently, and propose means for addressing this challenge.

Min Chen, Mateu Sbert, Alfie Abdul-Rahman et al.

Information theory can be used to analyze the cost-benefit of visualization processes. However, the current measure of benefit contains an unbounded term that is neither easy to estimate nor intuitive to interpret. In this work, we propose to revise the existing cost-benefit measure by replacing the unbounded term with a bounded one. We examine a number of bounded measures that include the Jenson-Shannon divergence and a new divergence measure formulated as part of this work. We use visual analysis to support the multi-criteria comparison, narrowing the search down to those options with better mathematical properties. We apply those remaining options to two visualization case studies to instantiate their uses in practical scenarios, while the collected real world data further informs the selection of a bounded measure, which can be used to estimate the benefit of visualization.

Alfie Abdul-Rahman, Rita Borgo, Min Chen et al.

Empirical studies form an integral part of visualization research. Not only can they facilitate the evaluation of various designs, techniques, systems, and practices in visualization, but they can also enable the discovery of the causalities explaining why and how visualization works. This state-of-the-art report focuses on controlled and semi-controlled empirical studies conducted in laboratories and crowd-sourcing environments. In particular, the survey provides a taxonomic analysis of over 129 empirical studies in the visualization literature. It juxtaposes these studies with topic developments between 1978 and 2017 in psychology, where controlled empirical studies have played a predominant role in research. To help appreciate this broad context, the paper provides two case studies in detail, where specific visualization-related topics were examined in the discipline of psychology as well as the field of visualization. Following a brief discussion on some latest developments in psychology, it outlines challenges and opportunities in making new discoveries about visualization through empirical studies.