Giulio Jacucci

Qijia Chen, Andrea Bellucci, Zhida Sun et al.

LLM-based mobile GUI agents treat every task invocation as an independent reasoning episode, requiring a full LLM inference call at each action step. This per-step dependence makes them stateless: a task completed successfully yesterday is re-derived from scratch today, with no improvement in reliability or speed. We present SkillDroid, a three-layer skill agent that compiles successful LLM-guided GUI trajectories into parameterized skill templates (sequences of UI actions with weighted element locators and typed parameter slots) and replays them on future invocations without any LLM calls. A matching cascade (regex patterns, embedding similarity, and app filtering) routes incoming instructions to stored skills, while a failure-learning layer triggers recompilation when skill reliability degrades. Over a 150-round longitudinal evaluation with systematic instruction variation and controlled perturbations, SkillDroid achieves an 85.3% success rate (23 percentage points above a stateless LLM baseline) while using 49% fewer LLM calls. The skill replay mechanism achieves a perfect 1000% success rate across 79 replay rounds at 2.4 times the speed of full LLM execution. Most critically, the system improves with use: its success rate converges upward from 87% to 91%, while the baseline degrades from 80% to 44%.

Qijia Chen, Andrea Bellucci, Giulio Jacucci

Newcomers are crucial for the growth of online communities, yet their successful integration into these spaces requires overcoming significant initial hurdles. Social Virtual Reality (VR) platforms are novel avenues that offer unprecedented online interaction experiences. Unlike well-studied two-dimensional online environments, the pathways to successful newcomer integration in online VR spaces are underexplored. Our research addresses this gap by examining the strategies used by newcomers to navigate early challenges in social VR and how they adapt. By focusing on active participants (ranging from newcomers currently navigating these hurdles to veterans who have successfully integrated) we isolate the specific strategies necessary for retention. We interviewed 24 active social VR users and conducted a reflexive thematic analysis. While participants identified barriers such as unfamiliar user interfaces, social norms, and overwhelming sensory input, our analysis reveals the adaptation strategies required to overcome them. Our findings expand on understanding newcomer persistence beyond traditional 2D environments, emphasizing how social dynamics influence the management of VR-specific issues like VR sickness during onboarding. Additionally, we highlight how successful newcomers overcome the lack of clear objectives in social VR by proactively constructing social meaning. We propose design suggestions to scaffold these successful integration pathways.

Marco Filetti, Hamed R. Tavakoli, Niklas Ravaja et al.

PeyeDF is a Portable Document Format (PDF) reader with eye tracking support, available as free and open source software. It is especially useful to researchers investigating reading and learning phenomena, as it integrates PDF reading-related behavioural data with gaze-related data. It is suitable for short and long-term research and supports multiple eye tracking systems. We utilised it to conduct an experiment which demonstrated that features obtained from both gaze and reading data collected in the past can predict reading comprehension which takes place in the future. PeyeDF also provides an integrated means for data collection and indexing using the DiMe personal data storage system. It is designed to collect data in the background without interfering with the reading experience, behaving like a modern lightweight PDF reader. Moreover, it supports annotations, tagging and collaborative work. A modular design allows the application to be easily modified in order to support additional eye tracking protocols and run controlled experiments. We discuss the implementation of the software and report on the results of the experiment which we conducted with it.

Barış Serim, Michiel Spapé, Giulio Jacucci

Researchers increasingly explore deploying brain-computer interfaces (BCIs) for able-bodied users, with the motivation of accessing mental states more directly than allowed by existing body-mediated interaction. This motivation seems to contradict the long-standing HCI emphasis on embodiment, namely the general claim that the body is crucial for cognition. This paper addresses this apparent contradiction through a review of insights from embodied cognition and interaction. We first critically examine the recent interest in BCIs and identify the extent cognition in the brain is integrated with the wider body as a central concern for research. We then define the implications of an integrated view of cognition for interface design and evaluation. A counterintuitive conclusion we draw is that embodiment per se should not imply a preference for body movements over brain signals. Yet it can nevertheless guide research by 1) providing body-grounded explanations for BCI performance, 2) proposing evaluation considerations that are neglected in modular views of cognition, and 3) through the direct transfer of its design insights to BCIs. We finally reflect on HCI's understanding of embodiment and identify the neural dimension of embodiment as hitherto overlooked.

Chen He, Luana Micallef, Barış Serim et al.

Exploratory search starts with ill-defined goals and involves browsing, learning, and formulating new targets for search. To fluidly support such dynamic search behaviours, we focus on devising interactive visual facets (IVF), visualising information facets to support user comprehension and control of the information space. To do this, we reviewed existing faceted search interfaces and derived two design requirements (DR) that have not been fully addressed to support fluid interactions in exploratory search. We then exemplified the requirements through devising an IVF tool, which coordinates a linear and a categorical facet representing the distribution and summarisation of items, respectively, and providing context for faceted exploration (DR1). To support rapid transitions between search criteria (DR2), the tool introduces a novel design concept of using facets to select items without filtering the item space. Particularly, we propose a filter-swipe technique that enables users to drag a categorical facet value sequentially over linear facet bars to view the items in the intersection of the two facets along with the categorical facet dynamically summarizing the items in the interaction. Three applications demonstrate how the features support information discovery with ease. A user study of 11 participants with realistic email search tasks shows that dynamic suggestions through the timeline navigation can help discover useful suggestions for search; the novel design concept was favoured over using facet values as filters. Based on these practices, we derive IVF design implications for fluid, exploratory searches.

Chen He, Luana Micallef, Liye He et al.

Understanding the quality of insight has become increasingly important with the trend of allowing users to post comments during visual exploration, yet approaches for qualifying insight are rare. This paper presents a case study to investigate the possibility of characterizing the quality of insight via the interactions performed. To do this, we devised the interaction of a visualization tool-MediSyn-for insight generation. MediSyn supports five types of interactions: selecting, connecting, elaborating, exploring, and sharing. We evaluated MediSyn with 14 participants by allowing them to freely explore the data and generate insights. We then extracted seven interaction patterns from their interaction logs and correlated the patterns to four aspects of insight quality. The results show the possibility of qualifying insights via interactions. Among other findings, exploration actions can lead to unexpected insights; the drill-down pattern tends to increase the domain values of insights. A qualitative analysis shows that using domain knowledge to guide exploration can positively affect the domain value of derived insights. We discuss the study's implications, lessons learned, and future research opportunities.

Luana Micallef, Iiris Sundin, Pekka Marttinen et al.

Providing accurate predictions is challenging for machine learning algorithms when the number of features is larger than the number of samples in the data. Prior knowledge can improve machine learning models by indicating relevant variables and parameter values. Yet, this prior knowledge is often tacit and only available from domain experts. We present a novel approach that uses interactive visualization to elicit the tacit prior knowledge and uses it to improve the accuracy of prediction models. The main component of our approach is a user model that models the domain expert's knowledge of the relevance of different features for a prediction task. In particular, based on the expert's earlier input, the user model guides the selection of the features on which to elicit user's knowledge next. The results of a controlled user study show that the user model significantly improves prior knowledge elicitation and prediction accuracy, when predicting the relative citation counts of scientific documents in a specific domain.

Manuel J. A. Eugster, Tuukka Ruotsalo, Michiel M. Spapé et al.

Finding relevant information from large document collections such as the World Wide Web is a common task in our daily lives. Estimation of a user's interest or search intention is necessary to recommend and retrieve relevant information from these collections. We introduce a brain-information interface used for recommending information by relevance inferred directly from brain signals. In experiments, participants were asked to read Wikipedia documents about a selection of topics while their EEG was recorded. Based on the prediction of word relevance, the individual's search intent was modeled and successfully used for retrieving new, relevant documents from the whole English Wikipedia corpus. The results show that the users' interests towards digital content can be modeled from the brain signals evoked by reading. The introduced brain-relevance paradigm enables the recommendation of information without any explicit user interaction, and may be applied across diverse information-intensive applications.