Bernhard Preim

Georgii Kostiuchik, Lalith Sharan, Benedikt Mayer et al.

Purpose: Machine learning models can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes of interest. Surgical workflow and instrument recognition tasks are complicated in this manner, because of heavy data imbalances resulting from different lengths of phases and their erratic occurrences. Furthermore, the issue becomes difficult as sub-properties that help define phases, like instrument (co-)occurrence, are usually not considered when defining the split. We argue that such sub-properties must be equally considered. Methods: This work presents a publicly available data visualization tool that enables interactive exploration of dataset splits for surgical phase and instrument recognition. It focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Particularly, it facilitates the assessment and identification of sub-optimal dataset splits. Results: We performed an analysis of common Cholec80 dataset splits using the proposed application and were able to uncover phase transitions and combinations of instruments that were not represented in one of the sets. Additionally, we outlined possible improvements to the splits. A user study with ten participants demonstrated the ability of participants to solve a selection of data exploration tasks using the proposed application. Conclusion: In highly unbalanced class distributions, special care should be taken with respect to the selection of an appropriate dataset split. Our interactive data visualization tool presents a promising approach for the assessment of dataset splits for surgical phase and instrument recognition. Evaluation results show that it can enhance the development of machine learning models. The application is available at https://cardio-ai.github.io/endovis-ml/ .

Sophia Dowlatabadi, Bernhard Preim, Monique Meuschke

In various fields, including medicine, age distributions are crucial. Despite widespread media coverage of health topics, there remains a need to enhance health communication. Narrative medical visualization is promising for improving information comprehension and retention. This study explores the most effective ways to present age distributions of diseases through narrative visualizations. We conducted a thorough analysis of existing visualizations, held workshops with a broad audience, and reviewed relevant literature. From this, we identified design choices focusing on comprehension, aesthetics, engagement, and memorability. We specifically tested three pictogram variants: pictograms as bars, stacked pictograms, and annotations. After evaluating 18 visualizations with 72 participants and three expert reviews, we determined that annotations were most effective for comprehension and aesthetics. However, traditional bar charts were preferred for engagement, and other variants were more memorable. The study provides a set of design recommendations based on these insights.

Apoorva Karagappa, Pawandeep Kaur Betz, Jonas Gilg et al.

As the world increasingly relies on mathematical models for forecasts in different areas, effective communication of uncertainty in time series predictions is important for informed decision making. This study explores how users estimate probabilistic uncertainty in time series predictions under different variants of line charts depicting uncertainty. It examines the role of individual characteristics and the influence of user-reported metrics on uncertainty estimations. By addressing these aspects, this paper aims to enhance the understanding of uncertainty visualization and for improving communication in time series forecast visualizations and the design of prediction data dashboards.As the world increasingly relies on mathematical models for forecasts in different areas, effective communication of uncertainty in time series predictions is important for informed decision making. This study explores how users estimate probabilistic uncertainty in time series predictions under different variants of line charts depicting uncertainty. It examines the role of individual characteristics and the influence of user-reported metrics on uncertainty estimations. By addressing these aspects, this paper aims to enhance the understanding of uncertainty visualization and for improving communication in time series forecast visualizations and the design of prediction data dashboards.

Anna Kleinau, Bernhard Preim, Monique Meuschke

In an era where black-box AI models are integral to decision-making across industries, robust methods for explaining these models are more critical than ever. While these models leverage complex feature interplay for accurate predictions, most explanation methods only assign relevance to individual features. There is a research gap in methods that effectively illustrate interactions between features, especially in visualizing higher-order interactions involving multiple features, which challenge conventional representation methods. To address this challenge in local explanations focused on individual instances, we employ a visual, subset-based approach to reveal relevant feature interactions. Our visual analytics tool FINCH uses coloring and highlighting techniques to create intuitive, human-centered visualizations, and provides additional views that enable users to calibrate their trust in the model and explanations. We demonstrate FINCH in multiple case studies, demonstrating its generalizability, and conducted an extensive human study with machine learning experts to highlight its helpfulness and usability. With this approach, FINCH allows users to visualize feature interactions involving any number of features locally.

Monique Meuschke, Laura Garrison, Noeska Smit et al.

Narrative visualization aims to communicate scientific results to a general audience and garners significant attention in various applications. Merging exploratory and explanatory visualization could effectively support a non-expert understanding of scientific processes. Medical research results, e.g., mechanisms of the healthy human body, explanations of pathological processes, or avoidable risk factors for diseases, are also interesting to a general audience that includes patients and their relatives. This paper discusses how narrative techniques can be applied to medical visualization to tell data-driven stories about diseases. We address the general public comprising people interested in medicine without specific medical background knowledge. We derived a general template for the narrative medical visualization of diseases. Applying this template to three diseases selected to span bone, vascular, and organ systems, we discuss how narrative techniques can support visual communication and facilitate understanding of medical data. Other scientists can adapt our proposed template to inform an audience on other diseases. With our work, we show the potential of narrative medical visualization and conclude with a comprehensive research agenda.

Patrick Saalfeld, Anna Schmeier, Wolfgang D'Hanis et al.

We introduce a Virtual Reality (VR) one-on-one tutoring system to support anatomy education. A student uses a fully immersive VR headset to explore the anatomy of the base of the human skull. A teacher guides the student by using the semi-immersive zSpace. Both systems are connected via network and each action is synchronized between both systems. The teacher is provided with various features to direct the student through the immersive learning experience. She can influence the student's navigation or provide annotations on the fly and, hereby, improve the students learning experience. The system is implemented using the \textit{Unity} game engine. A qualitative user study demonstrates that the one-on-one tutoring approach is feasible and sets a solid base for future research in the area of shared virtual environments for anatomy education.

Uli Niemann, Atrayee Neog, Benjamin Behrendt et al.

An accurate assessment of the cardiovascular system and prediction of cardiovascular diseases (CVDs) are crucial. Measured cardiac blood flow data provide insights about patient-specific hemodynamics, where many specialized techniques have been developed for the visual exploration of such data sets to better understand the influence of morphological and hemodynamic conditions on CVDs. However, there is a lack of machine learning approaches techniques that allow a feature-based classification of heart-healthy people and patients with CVDs. In this work, we investigate the potential of morphological and hemodynamic characteristics, extracted from measured blood flow data in the aorta, for the classification of heart-healthy volunteers and patients with bicuspid aortic valve (BAV). Furthermore, we research if there are characteristic features to classify male and female as well as older heart-healthy volunteers and BAV patients. We propose a data analysis pipeline for the classification of the cardiac status, encompassing feature selection, model training and hyperparameter tuning. In our experiments, we use several feature selection methods and classification algorithms to train separate models for the healthy subgroups and BAV patients. We report on classification performance and investigate the predictive power of morphological and hemodynamic features with regard to the classification of the defined groups. Finally, we identify the key features for the best models.

Lisa Schneider, Annika Niemann, Oliver Beuing et al.

Background and objective: MeshCNN is a recently proposed Deep Learning framework that drew attention due to its direct operation on irregular, non-uniform 3D meshes. On selected benchmarking datasets, it outperformed state-of-the-art methods within classification and segmentation tasks. Especially, the medical domain provides a large amount of complex 3D surface models that may benefit from processing with MeshCNN. However, several limitations prevent outstanding performances of MeshCNN on highly diverse medical surface models. Within this work, we propose MedMeshCNN as an expansion for complex, diverse, and fine-grained medical data. Methods: MedMeshCNN follows the functionality of MeshCNN with a significantly increased memory efficiency that allows retaining patient-specific properties during the segmentation process. Furthermore, it enables the segmentation of pathological structures that often come with highly imbalanced class distributions. Results: We tested the performance of MedMeshCNN on a complex part segmentation task of intracranial aneurysms and their surrounding vessel structures and reached a mean Intersection over Union of 63.24\%. The pathological aneurysm is segmented with an Intersection over Union of 71.4\%. Conclusions: These results demonstrate that MedMeshCNN enables the application of MeshCNN on complex, fine-grained medical surface meshes. The imbalanced class distribution deriving from the pathological finding is considered by MedMeshCNN and patient-specific properties are mostly retained during the segmentation process.

Bernhard Preim, Paul Klemm, Helwig Hauser et al.

Epidemiology characterizes the influence of causes to disease and health conditions of defined populations. Cohort studies are population-based studies involving usually large numbers of randomly selected individuals and comprising numerous attributes, ranging from self-reported interview data to results from various medical examinations, e.g., blood and urine samples. Since recently, medical imaging has been used as an additional instrument to assess risk factors and potential prognostic information. In this chapter, we discuss such studies and how the evaluation may benefit from visual analytics. Cluster analysis to define groups, reliable image analysis of organs in medical imaging data and shape space exploration to characterize anatomical shapes are among the visual analytics tools that may enable epidemiologists to fully exploit the potential of their huge and complex data. To gain acceptance, visual analytics tools need to complement more classical epidemiologic tools, primarily hypothesis-driven statistical analysis.

Anna Vilanova, Bernhard Preim, Roy van Pelt et al.

Morphology of cardiovascular tissue is influenced by the unsteady behavior of the blood flow and vice versa. Therefore, the pathogenesis of several cardiovascular diseases is directly affected by the blood-flow dynamics. Understanding flow behavior is of vital importance to understand the cardiovascular system and potentially harbors a considerable value for both diagnosis and risk assessment. The analysis of hemodynamic characteristics involves qualitative and quantitative inspection of the blood-flow field. Visualization plays an important role in the qualitative exploration, as well as the definition of relevant quantitative measures and its validation. There are two main approaches to obtain information about the blood flow: simulation by computational fluid dynamics, and in-vivo measurements. Although research on blood flow simulation has been performed for decades, many open problems remain concerning accuracy and patient-specific solutions. Possibilities for real measurement of blood flow have recently increased considerably by new developments in magnetic resonance imaging which enable the acquisition of 3D quantitative measurements of blood-flow velocity fields. This chapter presents the visualization challenges for both simulation and real measurements of unsteady blood-flow fields.