Bahador Saket

Jack Krolik, Herprit Mahal, Feroz Ahmad et al.

This paper explores the potential of using Large Language Models (LLMs) to automate the evaluation of responses in medical Question and Answer (Q\&A) systems, a crucial form of Natural Language Processing. Traditionally, human evaluation has been indispensable for assessing the quality of these responses. However, manual evaluation by medical professionals is time-consuming and costly. Our study examines whether LLMs can reliably replicate human evaluations by using questions derived from patient data, thereby saving valuable time for medical experts. While the findings suggest promising results, further research is needed to address more specific or complex questions that were beyond the scope of this initial investigation.

Hayeong Song, Yu Fu, Bahador Saket et al.

When performing data analysis, people often confront data sets containing missing values. We conducted an empirical study to understand the effects of visualizing those missing values on participants' decision-making processes while performing a visual data exploration task. More specifically, our study participants purchased a hypothetical portfolio of stocks based on a dataset where some stocks had missing values for attributes such as PE ratio, beta, and EPS. The experiment used scatterplots to communicate the stock data. For one group of participants, stocks with missing values simply were not shown, while the second group saw such stocks depicted with estimated values as points with error bars. We measured participants' cognitive load involved in decision-making with data with missing values. Our results indicate that their decision-making workflow was different across two conditions.

V. Das Swain, H. Kwon, S. Sargolzaei et al.

A comprehensive understanding of collocation can help understand performance outcomes. For university cohorts, this needs data that describes large groups over a long period. Harnessing user devices to infer this, while tempting, is challenged by privacy concerns, power consumption, and maintenance issues. Alternatively, embedding new sensors in the environment is limited by the expense of covering the entire campus. We investigate the feasibility of leveraging WiFi association logs for this purpose. While these provide coarse approximations of location, these are easily obtainable and depict multiple users on campus over a semester. We explore how these coarse collocations are related to individual performance. Specifically, we inspect the association between individual performance and the collocation behaviors of project group members. We study 163 students (in 54 project groups) over 14 weeks. After describing how we determine collocation with the WiFi logs, we present a study to analyze how collocation within groups relates to a student's final score. We find collocation behaviors show a significant correlation (Pearson's r = 0.24) with performance -- better than both peer feedback or individual behaviors like attendance. Finally, we discuss how repurposing WiFi logs can facilitate applications for domains like mental wellbeing and physical health.

Bahador Saket, Subhajit Das, Bum Chul Kwon et al.

Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study.

Bahador Saket, Samuel Huron, Charles Perin et al.

We investigate direct manipulation of graphical encodings as a method for interacting with visualizations. There is an increasing interest in developing visualization tools that enable users to perform operations by directly manipulating graphical encodings rather than external widgets such as checkboxes and sliders. Designers of such tools must decide which direct manipulation operations should be supported, and identify how each operation can be invoked. However, we lack empirical guidelines for how people convey their intended operations using direct manipulation of graphical encodings. We address this issue by conducting a qualitative study that examines how participants perform 15 operations using direct manipulation of standard graphical encodings. From this study, we 1) identify a list of strategies people employ to perform each operation, 2) observe commonalities in strategies across operations, and 3) derive implications to help designers leverage direct manipulation of graphical encoding as a method for user interaction.

Bahador Saket, Lei Jiang, Charles Perin et al.

Visualization tools usually leverage a single interaction paradigm (e.g., manual view specification, visualization by demonstration, etc.), which fosters the process of visualization construction. A large body of work has investigated the effectiveness of individual interaction paradigms, building an understanding of advantages and disadvantages of each in isolation. However, how can we leverage the benefits of multiple interaction paradigms by combining them into a single tool? We currently lack a holistic view of how interaction paradigms that use the same input modality (e.g., mouse) can be combined into a single tool and how people use such tools. To investigate opportunities and challenges in combining paradigms, we first created a multi-paradigm prototype (Liger) that combines two mouse-based interaction paradigms (manual view specification and visualization by demonstration) in a unified tool. We then conducted an exploratory study with Liger, providing initial evidence that people 1) use both paradigms interchangeably, 2) seamlessly switch between paradigms based on the operation at hand, and 3) choose to successfully complete a single operation using a combination of both paradigms.

Bahador Saket, Dominik Moritz, Halden Lin et al.

In this paper, we describe a research agenda for deriving design principles directly from data. We argue that it is time to go beyond manually curated and applied visualization design guidelines. We propose learning models of visualization design from data collected using graphical perception studies and build tools powered by the learned models. To achieve this vision, we need to 1) develop scalable methods for collecting training data, 2) collect different forms of training data, 3) advance interpretability of machine learning models, and 4) develop adaptive models that evolve as more data becomes available.

Bahador Saket, Alex Endert

We present an exploratory study comparing the visualization construction and data exploration processes of people using two visualization tools, each implementing a different interaction paradigm. One of the visualization tools implements the manual view specification paradigm (Polestar) and another implements the visualization by demonstration paradigm (VisExemplar). Findings of our study indicate that the interaction paradigms implemented in these tools influence: 1) approaches used for constructing visualizations, 2) how users form goals, 3) how many visualization alternatives are considered and created, and 4) the feeling of control during the visualization construction process.

Bahador Saket, Alex Endert, Cagatay Demiralp

Visualizations of tabular data are widely used; understanding their effectiveness in different task and data contexts is fundamental to scaling their impact. However, little is known about how basic tabular data visualizations perform across varying data analysis tasks and data attribute types. In this paper, we report results from a crowdsourced experiment to evaluate the effectiveness of five visualization types --- Table, Line Chart, Bar Chart, Scatterplot, and Pie Chart --- across ten common data analysis tasks and three data attribute types using two real-world datasets. We found the effectiveness of these visualization types significantly varies across task and data attribute types, suggesting that visualization design would benefit from considering context dependent effectiveness. Based on our findings, we derive recommendations on which visualizations to choose based on different tasks. We finally train a decision tree on the data we collected to drive a recommender, showcasing how to effectively engineer experimental user data into practical visualization systems.

Taeheon Kim, Bahador Saket, Alex Endert et al.

Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to being static (e.g., visualizations on presentation slides or posters). We believe in many of these cases, static visualizations will benefit from allowing users to perform interactive tasks on them. Inspired by the introduction of numerous commercial personal augmented reality (AR) devices, we propose an AR solution that allows interactive data exploration of datasets on static visualizations. In particular, we present a prototype system named VisAR that uses the Microsoft Hololens to enable users to complete interactive tasks on static visualizations.

Bahador Saket, Carlos Scheidegger, Stephen Kobourov

Traditionally, evaluation studies in information visualization have measured effectiveness by assessing performance time and accuracy. More recently, there has been a concerted effort to understand aspects beyond time and errors. In this paper we study enjoyment, which, while arguably not the primary goal of visualization, has been shown to impact performance and memorability. Different models of enjoyment have been proposed in psychology, education and gaming; yet there is no standard approach to evaluate and measure enjoyment in visualization. In this paper we relate the flow model of Csikszentmihalyi to Munzner's nested model of visualization evaluation and previous work in the area. We suggest that, even though previous papers tackled individual elements of flow, in order to understand what specifically makes a visualization enjoyable, it might be necessary to measure all specific elements.

Bahador Saket, Paolo Simonetto, Stephen Kobourov et al.

Effectively showing the relationships between objects in a dataset is one of the main tasks in information visualization. Typically there is a well-defined notion of distance between pairs of objects, and traditional approaches such as principal component analysis or multi-dimensional scaling are used to place the objects as points in 2D space, so that similar objects are close to each other. In another typical setting, the dataset is visualized as a network graph, where related nodes are connected by links. More recently, datasets are also visualized as maps, where in addition to nodes and links, there is an explicit representation of groups and clusters. We consider these three Techniques, characterized by a progressive increase of the amount of encoded information: node diagrams, node-link diagrams and node-link-group diagrams. We assess these three types of diagrams with a controlled experiment that covers nine different tasks falling broadly in three categories: node-based tasks, network-based tasks and group-based tasks. Our findings indicate that adding links, or links and group representations, does not negatively impact performance (time and accuracy) of node-based tasks. Similarly, adding group representations does not negatively impact the performance of network-based tasks. Node-link-group diagrams outperform the others on group-based tasks. These conclusions contradict results in other studies, in similar but subtly different settings. Taken together, however, such results can have significant implications for the design of standard and domain specific visualizations tools.

Bahador Saket, Paolo Simonetto, Stephen Kobourov

Task taxonomies for graph and network visualizations focus on tasks commonly encountered when analyzing graph connectivity and topology. However, in many application fields such as the social sciences (social networks), biology (protein interaction models), software engineering (program call graphs), connectivity and topology information is intertwined with group, clustering, and hierarchical information. Several recent visualization techniques, such as BubbleSets, LineSets and GMap, make explicit use of grouping and clustering, but evaluating such visualization has been difficult due to the lack of standardized group-level tasks. With this in mind, our goal is to define a new set of tasks that assess group-level comprehension. We propose several types of group-level tasks and provide several examples of each type. Finally, we characterize some of the proposed tasks using the multi-level typology of abstract visualization tasks. We believe that adding group-level tasks to the task taxonomy for graph visualization would make the taxonomy more useful for the recent graph visualization techniques. It would help evaluators define and categorize new tasks, and it would help generalize individual results collected in controlled experiments.