Yalong Yang

Yoonsang Kim, Yalong Yang, Arie E. Kaufman

We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Upon detection of similar or identical spatiotemporal activity, Memento proactively recalls user interests and delivers up-to-date responses through AR, seamlessly integrating AR experience into their daily routine. Unlike prior work, each interaction in Memento is not a transient event, but a connected series of interactions with coherent long--term perspective, tailored to the user's broader multimodal (visual, spatial, temporal, and embodied) context. We conduct a preliminary evaluation through user feedbacks with participants of diverse expertise in immersive apps, and explore the value of proactive context-aware AR assistant in everyday settings. We share our findings and challenges in designing a proactive, context-aware AR system.

Matthew Brehmer, Maxime Cordeil, Christophe Hurter et al.

We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

Yoonsang Kim, Devshree Jadeja, Divyansh Pradhan et al.

Speaking aloud to a wearable AR assistant in public can be socially awkward, and re-articulating the same requests every day creates unnecessary effort. We present SpeechLess, a wearable AR assistant that introduces a speech-based intent granularity control paradigm grounded in personalized spatial memory. SpeechLess helps users "speak less," while still obtaining the information they need, and supports gradual explicitation of intent when more complex expression is required. SpeechLess binds prior interactions to multimodal personal context-space, time, activity, and referents-to form spatial memories, and leverages them to extrapolate missing intent dimensions from under-specified user queries. This enables users to dynamically adjust how explicitly they express their informational needs, from full-utterance to micro/zero-utterance interaction. We motivate our design through a week-long formative study using a commercial smart glasses platform, revealing discomfort with public voice use, frustration with repetitive speech, and hardware constraints. Building on these insights, we design SpeechLess, and evaluate it through controlled lab and in-the-wild studies. Our results indicate that regulated speech-based interaction, can improve everyday information access, reduce articulation effort, and support socially acceptable use without substantially degrading perceived usability or intent resolution accuracy across diverse everyday environments.

Kun-Ting Chen, Tim Dwyer, Yalong Yang et al.

There are many methods for projecting spherical maps onto the plane. Interactive versions of these projections allow the user to centre the region of interest. However, the effects of such interaction have not previously been evaluated. In a study with 120 participants we find interaction provides significantly more accurate area, direction and distance estimation in such projections. The surface of 3D sphere and torus topologies provides a continuous surface for uninterrupted network layout. But how best to project spherical network layouts to 2D screens has not been studied, nor have such spherical network projections been compared to torus projections. Using the most successful interactive sphere projections from our first study, we compare spherical, standard and toroidal layouts of networks for cluster and path following tasks with 96 participants, finding benefits for both spherical and toroidal layouts over standard network layouts in terms of accuracy for cluster understanding tasks.

Tica Lin, Yalong Yang, Johanna Beyer et al.

Augmented Reality (AR) embeds digital information into objects of the physical world. Data can be shown in-situ, thereby enabling real-time visual comparisons and object search in real-life user tasks, such as comparing products and looking up scores in a sports game. While there have been studies on designing AR interfaces for situated information retrieval, there has only been limited research on AR object labeling for visual search tasks in the spatial environment. In this paper, we identify and categorize different design aspects in AR label design and report on a formal user study on labels for out-of-view objects to support visual search tasks in AR. We design three visualization techniques for out-of-view object labeling in AR, which respectively encode the relative physical position (height-encoded), the rotational direction (angle-encoded), and the label values (value-encoded) of the objects. We further implement two traditional in-view object labeling techniques, where labels are placed either next to the respective objects (situated) or at the edge of the AR FoV (boundary). We evaluate these five different label conditions in three visual search tasks for static objects. Our study shows that out-of-view object labels are beneficial when searching for objects outside the FoV, spatial orientation, and when comparing multiple spatially sparse objects. Angle-encoded labels with directional cues of the surrounding objects have the overall best performance with the highest user satisfaction. We discuss the implications of our findings for future immersive AR interface design.

Johanna Beyer, Yalong Yang, Hanspeter Pfister

We present how to integrate Design Sprints and project-based learning into introductory visualization courses. A design sprint is a unique process based on rapid prototyping and user testing to define goals and validate ideas before starting costly development. The well-defined, interactive, and time-constrained design cycle makes design sprints a promising option for teaching project-based and active-learning-centered courses to increase student engagement and hands-on experience. Over the past five years, we have adjusted the design sprint methodology for teaching a range of visualization courses. We present a detailed guide on incorporating design sprints into large undergraduate and small professional development courses in both online and on-campus settings. Design sprint results, including quantitative and qualitative student feedback, show that design sprints engage students and help practice and apply visualization and design skills. We provide design sprint teaching materials, show examples of student-created work, and discuss limitations and lessons learned.

Tica Lin, Rishi Singh, Yalong Yang et al.

We present an observational study to compare co-located and situated real-time visualizations in basketball free-throw training. Our goal is to understand the advantages and concerns of applying immersive visualization to real-world skill-based sports training and to provide insights for designing AR sports training systems. We design both a situated 3D visualization on a head-mounted display and a 2D visualization on a co-located display to provide immediate visual feedback on a player's shot performance. Using a within-subject study design with experienced basketball shooters, we characterize user goals, report on qualitative training experiences, and compare the quantitative training results. Our results show that real-time visual feedback helps athletes refine subsequent shots. Shooters in our study achieve greater angle consistency with our visual feedback. Furthermore, AR visualization promotes an increased focus on body form in athletes. Finally, we present suggestions for the design of future sports AR studies.

Sarah Schöttler, Yalong Yang, Hanspeter Pfister et al.

This paper surveys visualization and interaction techniques for geospatial networks from a total of 95 papers. Geospatial networks are graphs where nodes and links can be associated with geographic locations. Examples can include social networks, trade and migration, as well as traffic and transport networks. Visualizing geospatial networks poses numerous challenges around the integration of both network and geographical information as well as additional information such as node and link attributes, time, and uncertainty. Our overview analyzes existing techniques along four dimensions: i) the representation of geographical information, ii) the representation of network information, iii) the visual integration of both, and iv) the use of interaction. These four dimensions allow us to discuss techniques with respect to the trade-offs they make between showing information across all these dimensions and how they solve the problem of showing as much information as necessary while maintaining readability of the visualization. https://geonetworks.github.io.

Yalong Yang, Maxime Cordeil, Johanna Beyer et al.

Abstract data has no natural scale and so interactive data visualizations must provide techniques to allow the user to choose their viewpoint and scale. Such techniques are well established in desktop visualization tools. The two most common techniques are zoom+pan and overview+detail. However, how best to enable the analyst to navigate and view abstract data at different levels of scale in immersive environments has not previously been studied. We report the findings of the first systematic study of immersive navigation techniques for 3D scatterplots. We tested four conditions that represent our best attempt to adapt standard 2D navigation techniques to data visualization in an immersive environment while still providing standard immersive navigation techniques through physical movement and teleportation. We compared room-sized visualization versus a zooming interface, each with and without an overview. We find significant differences in participants' response times and accuracy for a number of standard visual analysis tasks. Both zoom and overview provide benefits over standard locomotion support alone (i.e., physical movement and pointer teleportation). However, which variation is superior, depends on the task. We obtain a more nuanced understanding of the results by analyzing them in terms of a time-cost model for the different components of navigation: way-finding, travel, number of travel steps, and context switching.

Vahan Yoghourdjian, Yalong Yang, Tim Dwyer et al.

Node-link diagrams are widely used to visualise networks. However, even the best network layout algorithms ultimately result in 'hairball' visualisations when the graph reaches a certain degree of complexity, requiring simplification through aggregation or interaction (such as filtering) to remain usable. Until now, there has been little data to indicate at what level of complexity node-link diagrams become ineffective or how visual complexity affects cognitive load. To this end, we conducted a controlled study to understand workload limits for a task that requires a detailed understanding of the network topology---finding the shortest path between two nodes. We tested performance on graphs with 25 to 175 nodes with varying density. We collected performance measures (accuracy and response time), subjective feedback, and physiological measures (EEG, pupil dilation, and heart rate variability). To the best of our knowledge, this is the first network visualisation study to include physiological measures. Our results show that people have significant difficulty finding the shortest path in high-density node-link diagrams with more than 50 nodes and even low-density graphs with more than 100 nodes. From our collected EEG data we observe functional differences in brain activity between hard and easy tasks. We found that cognitive load increased up to a certain level of difficulty after which it decreased, likely because participants had given up. We also explored the effects of global network layout features such as size or number of crossings, and features of the shortest path such as length or straightness on task difficulty. We found that global features generally had a greater impact than those of the shortest path.

Yalong Yang, Tim Dwyer, Kim Marriott et al.

We introduce Tilt Map, a novel interaction technique for intuitively transitioning between 2D and 3D map visualisations in immersive environments. Our focus is visualising data associated with areal features on maps, for example, population density by state. Tilt Map transitions from 2D choropleth maps to 3D prism maps to 2D bar charts to overcome the limitations of each. Our paper includes two user studies. The first study compares subjects' task performance interpreting population density data using 2D choropleth maps and 3D prism maps in virtual reality (VR). We observed greater task accuracy with prism maps, but faster response times with choropleth maps. The complementarity of these views inspired our hybrid Tilt Map design. Our second study compares Tilt Map to: a side-by-side arrangement of the various views; and interactive toggling between views. The results indicate benefits for Tilt Map in user preference; and accuracy (versus side-by-side) and time (versus toggle).

Tica Lin, Yalong Yang, Johanna Beyer et al.

We present our initial investigation of key challenges and potentials of immersive analytics (IA) in sports, which we call SportsXR. Sports are usually highly dynamic and collaborative by nature, which makes real-time decision making ubiquitous. However, there is limited support for athletes and coaches to make informed and clear-sighted decisions in real-time. SportsXR aims to support situational awareness for better and more agile decision making in sports. In this paper, we identify key challenges in SportsXR, including data collection, in-game decision making, situated sport-specific visualization design, and collaborating with domain experts. We then present potential user scenarios in training, coaching, and fan experiences. This position paper aims to inform and inspire future SportsXR research.

Yalong Yang, Kim Marriott, Matthew Butler et al.

Visualisations are commonly used to understand social, biological and other kinds of networks. Currently, we do not know how to effectively present network data to people who are blind or have low-vision (BLV). We ran a controlled study with 8 BLV participants comparing four tactile representations: organic node-link diagram, grid node-link diagram, adjacency matrix and braille list. We found that the node-link representations were preferred and more effective for path following and cluster identification while the matrix and list were better for adjacency tasks. This is broadly in line with findings for the corresponding visual representations.

Yalong Yang, Tim Dwyer, Bernhard Jenny et al.

Immersive virtual- and augmented-reality headsets can overlay a flat image against any surface or hang virtual objects in the space around the user. The technology is rapidly improving and may, in the long term, replace traditional flat panel displays in many situations. When displays are no longer intrinsically flat, how should we use the space around the user for abstract data visualisation? In this paper, we ask this question with respect to origin-destination flow data in a global geographic context. We report on the findings of three studies exploring different spatial encodings for flow maps. The first experiment focuses on different 2D and 3D encodings for flows on flat maps. We find that participants are significantly more accurate with raised flow paths whose height is proportional to flow distance but fastest with traditional straight line 2D flows. In our second and third experiment, we compared flat maps, 3D globes and a novel interactive design we call MapsLink, involving a pair of linked flat maps. We find that participants took significantly more time with MapsLink than other flow maps while the 3D globe with raised flows was the fastest, most accurate, and most preferred method. Our work suggests that careful use of the third spatial dimension can resolve visual clutter in complex flow maps.

Yalong Yang, Bernhard Jenny, Tim Dwyer et al.

This paper explores different ways to render world-wide geographic maps in virtual reality (VR). We compare: (a) a 3D exocentric globe, where the user's viewpoint is outside the globe; (b) a flat map (rendered to a plane in VR); (c) an egocentric 3D globe, with the viewpoint inside the globe; and (d) a curved map, created by projecting the map onto a section of a sphere which curves around the user. In all four visualisations the geographic centre can be smoothly adjusted with a standard handheld VR controller and the user, through a head-tracked headset, can physically move around the visualisation. For distance comparison, exocentric globe is more accurate than egocentric globe and flat map. For area comparison, more time is required with exocentric and egocentric globes than with flat and curved maps. For direction estimation, the exocentric globe is more accurate and faster than the other visual presentations. Our study participants had a weak preference for the exocentric globe. Generally, the curved map had benefits over the flat map. In almost all cases the egocentric globe was found to be the least effective visualisation. Overall, our results provide support for the use of exocentric globes for geographic visualisation in mixed-reality.

Yalong Yang, Tim Dwyer, Sarah Goodwin et al.

Showing flows of people and resources between multiple geographic locations is a challenging visualisation problem. We conducted two quantitative user studies to evaluate different visual representations for such dense many-to-many flows. In our first study we compared a bundled node-link flow map representation and OD Maps [37] with a new visualisation we call MapTrix. Like OD Maps, MapTrix overcomes the clutter associated with a traditional flow map while providing geographic embedding that is missing in standard OD matrix representations. We found that OD Maps and MapTrix had similar performance while bundled node-link flow map representations did not scale at all well. Our second study compared participant performance with OD Maps and MapTrix on larger data sets. Again performance was remarkably similar.

Linda Woodburn, Yalong Yang, Kim Marriott

We have compared three common visualisations for hierarchical quantitative data, treemaps, icicle plots and sunburst charts as well as a semicircular variant of sunburst charts we call the sundown chart. In a pilot study, we found that the sunburst chart was least preferred. In a controlled study with 12 participants, we compared treemaps, icicle plots and sundown charts. Treemap was the least preferred and had a slower performance on a basic navigation task and slower performance and accuracy in hierarchy understanding tasks. The icicle plot and sundown chart had similar performance with slight user preference for the icicle plot.

Yalong Yang

This thesis develops and evaluates effective techniques for visualisation of flows (e.g. of people, trade, knowledge) between places on geographic maps. This geographically-embedded flow data contains information about geographic locations, and flows from origin locations to destination locations. We explored the design space of OD flow visualisation in both 2D and immersive environments. We do so by creating novel OD flow visualisations in both environments, and then conducting controlled user studies to evaluate different designs.

Yalong Yang, Sarah Goodwin

In this paper, we present our analysis of five expert interviews, each from a different application domain. Such analysis is crucial to understanding the real-world scenarios of analysing geographically-embedded flow data. The results of our analysis show that similar high-level tasks were conducted in different domains. To better describe the targets of these tasks, we proposed three flow-targets for analysing geographically-embedded flow data: single flow, total flow and regional flow.