Torsten Möller

Steven Bergner, Michael Sedlmair, Sareh Nabi et al.

In this paper we introduce paraglide, a visualization system designed for interactive exploration of parameter spaces of multi-variate simulation models. To get the right parameter configuration, model developers frequently have to go back and forth between setting parameters and qualitatively judging the outcomes of their model. During this process, they build up a grounded understanding of the parameter effects in order to pick the right setting. Current state-of-the-art tools and practices, however, fail to provide a systematic way of exploring these parameter spaces, making informed decisions about parameter settings a tedious and workload-intensive task. Paraglide endeavors to overcome this shortcoming by assisting the sampling of the parameter space and the discovery of qualitatively different model outcomes. This results in a decomposition of the model parameter space into regions of distinct behaviour. We developed paraglide in close collaboration with experts from three different domains, who all were involved in developing new models for their domain. We first analyzed current practices of six domain experts and derived a set of design requirements, then engaged in a longitudinal user-centered design process, and finally conducted three in-depth case studies underlining the usefulness of our approach.

Aleksandar Doknic, Torsten Möller

Despite their effective use in various fields, many aspects of neural networks are poorly understood. One important way to investigate the characteristics of neural networks is to explore the loss landscape. However, most models produce a high-dimensional non-convex landscape which is difficult to visualize. We discuss and extend existing visualization methods based on 1D- and 2D slicing with a novel method that approximates the actual loss landscape geometry by using charts with interpretable axes. Based on the assumption that observations on small neural networks can generalize to more complex systems and provide us with helpful insights, we focus on small models in the range of a few dozen weights, which enables computationally cheap experiments and the use of an interactive dashboard. We observe symmetries around the zero vector, the influence of different layers on the global landscape, the different weight sensitivities around a minimizer, and how gradient descent navigates high-loss obstacles. The user study resulted in an average SUS (System Usability Scale) score with suggestions for improvement and opened up a number of possible application scenarios, such as autoencoders and ensemble networks.

Regina Schuster, Kathleen Gregory, Torsten Möller et al.

Public-facing data visualizations can play a vital role in making complex information clear and engaging, thereby encouraging informed public discourse and participation. However, existing work offers limited insight into how practitioners make design decisions based on their envisioned target audiences and across different media channels. To investigate this, we conducted semi-structured interviews with 21 professionals from journalistic settings, focusing on how they conceptualize their readers, translate these notions into design choices, and evaluate their work. We found that practitioners often rely on broad audience definitions, despite considering ``knowing their readers'' essential. Evaluation primarily relies on peer feedback or social metrics rather than user testing. From these accounts, we identify recurring strategies employed to reach general, often undefined publics. We discuss implications for audience-centered authoring tools, proposing features such as persona simulations and content-adaptive multi-format authoring, message-first rhetoric-aware workflows, and lightweight in-tool evaluation to better support the realities of public-facing design.

Laura Lotteraner, Anna Kurtenkova, Torsten Möller et al.

A range of charts with different strengths and weaknesses exists to support the visual analysis of univariate distributions, with a limited understanding of which charts best support which tasks and users, and how practitioners use charts. We categorize the available charts for univariate distributions into four groups and present the results of a mixed-methods comparison (n=215) of participants' perception and preferences across boxplots, violinplots, jittered stripplots, and histograms as representatives of their respective categories. The click-to-select approach in our study, combined with data on participants' subjective experiences and preferences, allows to both measure accuracy on benchmark tasks and discuss participants' choices qualitatively. Our analysis reveals differences between charts in task accuracy, common misunderstandings, and preferences across various low-level tasks, and indicates that chart preference and familiarity do not necessarily align with participants' task performance. Interviews with five visualization practitioners further reveal that charts widely preferred by general audiences (such as histograms) or commonly used in scientific domains (such as boxplots) are not inherently the most effective for all tasks.

Aleksandar Doknic, Torsten Möller

Machine learning-based classifiers are commonly evaluated by metrics like accuracy, but deeper analysis is required to understand their strengths and weaknesses. MLMC is a visual exploration tool that tackles the challenge of multi-label classifier comparison and evaluation. It offers a scalable alternative to confusion matrices which are commonly used for such tasks, but don't scale well with a large number of classes or labels. Additionally, MLMC allows users to view classifier performance from an instance perspective, a label perspective, and a classifier perspective. Our user study shows that the techniques implemented by MLMC allow for a powerful multi-label classifier evaluation while preserving user friendliness.

Timothée Schmude, Laura Koesten, Torsten Möller et al.

Every AI system that makes decisions about people has a group of stakeholders that are personally affected by these decisions. However, explanations of AI systems rarely address the information needs of this stakeholder group, who often are AI novices. This creates a gap between conveyed information and information that matters to those who are impacted by the system's decisions, such as domain experts and decision subjects. To address this, we present the "XAI Novice Question Bank," an extension of the XAI Question Bank containing a catalog of information needs from AI novices in two use cases: employment prediction and health monitoring. The catalog covers the categories of data, system context, system usage, and system specifications. We gathered information needs through task-based interviews where participants asked questions about two AI systems to decide on their adoption and received verbal explanations in response. Our analysis showed that participants' confidence increased after receiving explanations but that their understanding faced challenges. These included difficulties in locating information and in assessing their own understanding, as well as attempts to outsource understanding. Additionally, participants' prior perceptions of the systems' risks and benefits influenced their information needs. Participants who perceived high risks sought explanations about the intentions behind a system's deployment, while those who perceived low risks rather asked about the system's operation. Our work aims to support the inclusion of AI novices in explainability efforts by highlighting their information needs, aims, and challenges. We summarize our findings as five key implications that can inform the design of future explanations for lay stakeholder audiences.

Timothée Schmude, Laura Koesten, Torsten Möller et al.

We argue that explanations for "algorithmic decision-making" (ADM) systems can profit by adopting practices that are already used in the learning sciences. We shortly introduce the importance of explaining ADM systems, give a brief overview of approaches drawing from other disciplines to improve explanations, and present the results of our qualitative task-based study incorporating the "six facets of understanding" framework. We close with questions guiding the discussion of how future studies can leverage an interdisciplinary approach.

Raphael Sahann, Torsten Möller, Johanna Schmidt

This paper presents a quantitative user study to evaluate how well users can visually perceive the underlying data distribution from a histogram representation. We used different sample and bin sizes and four different distributions (uniform, normal, bimodal, and gamma). The study results confirm that, in general, more bins correlate with fewer errors by the viewers. However, upon a certain number of bins, the error rate cannot be improved by adding more bins. By comparing our study results with the outcomes of existing mathematical models for histogram binning (e.g., Sturges' formula, Scott's normal reference rule, the Rice Rule, or Freedman-Diaconis' choice), we can see that most of them overestimate the number of bins necessary to make the distribution visible to a human viewer.

Meng Ling, Jian Chen, Torsten Möller et al.

We present document domain randomization (DDR), the first successful transfer of convolutional neural networks (CNNs) trained only on graphically rendered pseudo-paper pages to real-world document segmentation. DDR renders pseudo-document pages by modeling randomized textual and non-textual contents of interest, with user-defined layout and font styles to support joint learning of fine-grained classes. We demonstrate competitive results using our DDR approach to extract nine document classes from the benchmark CS-150 and papers published in two domains, namely annual meetings of Association for Computational Linguistics (ACL) and IEEE Visualization (VIS). We compare DDR to conditions of style mismatch, fewer or more noisy samples that are more easily obtained in the real world. We show that high-fidelity semantic information is not necessary to label semantic classes but style mismatch between train and test can lower model accuracy. Using smaller training samples had a slightly detrimental effect. Finally, network models still achieved high test accuracy when correct labels are diluted towards confusing labels; this behavior hold across several classes.

Jian Chen, Meng Ling, Rui Li et al.

We present the VIS30K dataset, a collection of 29,689 images that represents 30 years of figures and tables from each track of the IEEE Visualization conference series (Vis, SciVis, InfoVis, VAST). VIS30K's comprehensive coverage of the scientific literature in visualization not only reflects the progress of the field but also enables researchers to study the evolution of the state-of-the-art and to find relevant work based on graphical content. We describe the dataset and our semi-automatic collection process, which couples convolutional neural networks (CNN) with curation. Extracting figures and tables semi-automatically allows us to verify that no images are overlooked or extracted erroneously. To improve quality further, we engaged in a peer-search process for high-quality figures from early IEEE Visualization papers. With the resulting data, we also contribute VISImageNavigator (VIN, visimagenavigator.github.io), a web-based tool that facilitates searching and exploring VIS30K by author names, paper keywords, title and abstract, and years.

Florian Heimerl, Christoph Kralj, Torsten Möller et al.

This paper introduces embComp, a novel approach for comparing two embeddings that capture the similarity between objects, such as word and document embeddings. We survey scenarios where comparing these embedding spaces is useful. From those scenarios, we derive common tasks, introduce visual analysis methods that support these tasks, and combine them into a comprehensive system. One of embComp's central features are overview visualizations that are based on metrics for measuring differences in the local structure around objects. Summarizing these local metrics over the embeddings provides global overviews of similarities and differences. Detail views allow comparison of the local structure around selected objects and relating this local information to the global views. Integrating and connecting all of these components, embComp supports a range of analysis workflows that help understand similarities and differences between embedding spaces. We assess our approach by applying it in several use cases, including understanding corpora differences via word vector embeddings, and understanding algorithmic differences in generating embeddings.