Po-Ming Law

Po-Ming Law, Leo Yu-Ho Lo, Alex Endert et al.

Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impact of showing different information on user perceptions of a question-answering system. We found that in a system that occasionally provided unreasonable responses, showing a scatterplot increased the plausibility of unreasonable causal claims. Also, simply warning participants that correlation is not causation seemed to lead participants to accept reasonable causal claims more cautiously. We observed a strong tendency among participants to associate correlation with causation. Yet, the warning appeared to reduce the tendency. Grounded in the findings, we propose ways to reduce the illusion of causality when using question-answering systems.

Po-Ming Law, Alex Endert, John Stasko

Many researchers have explored tools that aim to recommend data insights to users. These tools automatically communicate a rich diversity of data insights and offer such insights for many different purposes. However, there is a lack of structured understanding concerning what researchers of these tools mean by "insight" and what tasks in the analysis workflow these tools aim to support. We conducted a systematic review of existing systems that seek to recommend data insights. Grounded in the review, we propose 12 types of automated insights and four purposes of automating insights. We further discuss the design opportunities emerged from our analysis.

Po-Ming Law, Alex Endert, John Stasko

While many visualization researchers have attempted to define data insights, little is known about how visualization users perceive them. We interviewed 23 professional users of end-user visualization platforms (e.g., Tableau and Power BI) about their experiences with data insights. We report on seven characteristics of data insights based on interviewees' descriptions. Grounded in these characteristics, we propose practical implications for creating tools that aim to automatically communicate data insights to users.

Po-Ming Law, Sana Malik, Fan Du et al.

While decision makers have begun to employ machine learning, machine learning models may make predictions that bias against certain demographic groups. Semi-automated bias detection tools often present reports of automatically-detected biases using a recommendation list or visual cues. However, there is a lack of guidance concerning which presentation style to use in what scenarios. We conducted a small lab study with 16 participants to investigate how presentation style might affect user behaviors in reviewing bias reports. Participants used both a prototype with a recommendation list and a prototype with visual cues for bias detection. We found that participants often wanted to investigate the performance measures that were not automatically detected as biases. Yet, when using the prototype with a recommendation list, they tended to give less consideration to such measures. Grounded in the findings, we propose information load and comprehensiveness as two axes for characterizing bias detection tasks and illustrate how the two axes could be adopted to reason about when to use a recommendation list or visual cues.

Po-Ming Law, Sana Malik, Fan Du et al.

Machine learning models often make predictions that bias against certain subgroups of input data. When undetected, machine learning biases can constitute significant financial and ethical implications. Semi-automated tools that involve humans in the loop could facilitate bias detection. Yet, little is known about the considerations involved in their design. In this paper, we report on an interview study with 11 machine learning practitioners for investigating the needs surrounding semi-automated bias detection tools. Based on the findings, we highlight four considerations in designing to guide system designers who aim to create future tools for bias detection.

Po-Ming Law, Yanhong Wu, Rahul C. Basole

Getting the overall picture of how a large number of ego-networks evolve is a common yet challenging task. Existing techniques often require analysts to inspect the evolution patterns of ego-networks one after another. In this study, we explore an approach that allows analysts to interactively create spatial layouts in which each dot is a dynamic ego-network. These spatial layouts provide overviews of the evolution patterns of ego-networks, thereby revealing different global patterns such as trends, clusters and outliers in evolution patterns. To let analysts interactively construct interpretable spatial layouts, we propose a data transformation pipeline, with which analysts can adjust the spatial layouts and convert dynamic egonetworks into event sequences to aid interpretations of the spatial positions. Based on this transformation pipeline, we developed Segue, a visual analysis system that supports thorough exploration of the evolution patterns of ego-networks. Through two usage scenarios, we demonstrate how analysts can gain insights into the overall evolution patterns of a large collection of ego-networks by interactively creating different spatial layouts.