Tiago Guerreiro

Pavithren V S Pakianathan, Rania Islambouli, Diogo Branco et al.

Individuals are increasingly generating substantial personal health and lifestyle data, e.g. through wearables and smartphones. While such data could transform preventative care, its integration into clinical practice is hindered by its scale, heterogeneity and the time pressure and data literacy of healthcare professionals (HCPs). We explore how large language models (LLMs) can support sensemaking of patient-generated health data (PGHD) with automated summaries and natural language data exploration. Using cardiovascular disease (CVD) risk reduction as a use case, 16 HCPs reviewed multimodal PGHD in a mixed-methods study with a prototype that integrated common charts, LLM-generated summaries, and a conversational interface. Findings show that AI summaries provided quick overviews that anchored exploration, while conversational interaction supported flexible analysis and bridged data-literacy gaps. However, HCPs raised concerns about transparency, privacy, and overreliance. We contribute empirical insights and sociotechnical design implications for integrating AI-driven summarization and conversation into clinical workflows to support PGHD sensemaking.

Sérgio Alves, Carlos Duarte, Kyle Montague et al.

User interface personalization enhances digital efficiency, usability, and accessibility. However, in user-driven setups, limited support for identifying and evaluating worthwhile opportunities often leads to underuse. We explore a reflexive personalization approach where individuals engage with their digital interaction data to identify meaningful personalization opportunities and benefits. We interviewed 12 participants, using experimental vignettes as design probes to support reflection on different forms of using interaction data to empower decision-making in personalization and the preferred level of system support. We found that people can independently identify personalization opportunities but prefer system support through visual personalization suggestions. Interaction data can shape how users perceive and approach personalization by reinforcing the perceived value of change and data collection, helping them weigh benefits against effort, and increasing the transparency of system suggestions. We discuss opportunities for designing personalization software that raises end-users' agency over interfaces through reflective engagement with their interaction data.

Susana Nunes, Tiago Guerreiro, Catia Pesquita

AI explanation methods often assume a static user model, producing non-adaptive explanations regardless of expert goals, reasoning strategies, or decision contexts. Knowledge graph-based explanations, despite their capacity for grounded, path-based reasoning, inherit this limitation. In complex domains such as scientific discovery, this assumption fails to capture the diversity of cognitive strategies and epistemic stances among experts, preventing explanations that foster deeper understanding and informed decision-making. However, the scarcity of human experts limits the use of direct human feedback to produce adaptive explanations. We present a reinforcement learning approach for scientific explanation generation that incorporates agentic personas, structured representations of expert reasoning strategies, that guide the explanation agent towards specific epistemic preferences. In an evaluation of knowledge graph-based explanations for drug discovery, we tested two personas that capture distinct epistemic stances derived from expert feedback. Results show that persona-driven explanations match state-of-the-art predictive performance while persona preferences closely align with those of their corresponding experts. Adaptive explanations were consistently preferred over non-adaptive baselines (n = 22), and persona-based training reduces feedback requirements by two orders of magnitude. These findings demonstrate how agentic personas enable scalable adaptive explainability for AI systems in complex and high-stakes domains.

André Rodrigues, André Santos, Kyle Montague et al.

Touch data, and in particular text-entry data, has been mostly collected in the laboratory, under controlled conditions. While touch and text-entry data have consistently shown its potential for monitoring and detecting a variety of conditions and impairments, its deployment in-the-wild remains a challenge. In this paper, we present WildKey, an Android keyboard toolkit that allows for the usable deployment of in-the-wild user studies. WildKey is able to analyze text-entry behaviors through implicit and explicit text-entry data collection while ensuring user privacy. We detail each of the WildKey's components and features, all of the metrics collected, and discuss the steps taken to ensure user privacy and promote compliance.

Letícia Seixas Pereira, José Coelho, André Rodrigues et al.

User-generated content plays a key role in social networking, allowing a more active participation, socialisation, and collaboration among users. In particular, media content has been gaining a lot of ground, allowing users to express themselves through different types of formats such as images, GIFs and videos. The majority of this growing type of online content remains inaccessible to a part of the population, despite available tools to mitigate this source of exclusion. We sought to understand how people are perceiving these online contents in their networks and how to support tools are being used. To do so, we performed an online survey of 258 social network users and a follow-up interview conducted with 20 of them - 7 of them self-reporting blind and 13 sighted users without a disability. Results show how the different approaches being employed by major platforms are still not sufficient to properly address this issue. Our findings reveal that mainstream users are not aware of the possibility and the benefits of adopting accessible practices. From the general perspectives of end-users experiencing accessible practices, concerning barriers encountered, and motivational factors, we also discuss further approaches to create more user engagement and awareness.

André Rodrigues, André Santos, Kyle Montague et al.

Accessibility assessments typically focus on determining a binary measurement of task performance success/failure; and often neglect to acknowledge the nuances of those interactions. Although a large population of blind people find smartphone interactions possible, many experiences take a significant toll and can have a lasting negative impact on the individual and their willingness to step out of technological comfort zones. There is a need to assist and support individuals with the adoption and learning process of new tasks to mitigate these negative experiences. We contribute with a human-powered nonvisual task assistant for smartphones to provide pervasive assistance. We argue, in addition to success, one must carefully consider promoting and evaluating factors such as self-efficacy and the belief in one's own abilities to control and learn to use technology. In this paper, we show effective assistant positively affects self-efficacy when performing new tasks with smartphones, affects perceptions of accessibility and enables systemic task-based learning.

David Gonçalves, André Rodrigues, Mike L. Richardson et al.

The landscape of digital games is segregated by player ability. For example, sighted players have a multitude of highly visual games at their disposal, while blind players may choose from a variety of audio games. Attempts at improving cross-ability access to any of those are often limited in the experience they provide, or disregard multiplayer experiences. We explore ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. Our team designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.

André Rodrigues, Hugo Nicolau, Kyle Montague et al.

Blind people face significant challenges when using smartphones. The focus on improving non-visual mobile accessibility has been at the level of touchscreen access. Our research investigates the challenges faced by blind people in their everyday usage of mobile phones. In this paper, we present a set of studies performed with the target population, novices and experts, using a variety of methods, targeted at identifying and verifying challenges; and coping mechanisms. Through a multiple methods approach we identify and validate challenges locally with a diverse set of user expertise and devices, and at scale through the analyses of the largest Android and iOS dedicate forums for blind people. We contribute with a prioritized corpus of smartphone challenges for blind people, and a discussion on a set of directions for future research that tackle the open and often overlooked challenges.

Diogo Marques, Luís Carriço, Tiago Guerreiro

As people store more personal data in their smartphones, the consequences of having it stolen or lost become an increasing concern. A typical counter-measure to avoid this risk is to set up a secret code that has to be entered to unlock the device after a period of inactivity. However, for blind users, PINs and passwords are inadequate, since entry 1) consumes a non-trivial amount of time, e.g. using screen readers, 2) is susceptible to observation, where nearby people can see or hear the secret code, and 3) might collide with social norms, e.g. disrupting personal interactions. Tap-based authentication methods have been presented and allow unlocking to be performed in a short time and support naturally occurring inconspicuous behavior (e.g. concealing the device inside a jacket) by being usable with a single hand. This paper presents a study with blind users (N = 16) where an authentication method based on tap phrases is evaluated. Results showed the method to be usable and to support the desired inconspicuity.

Tiago Guerreiro, Hugo Nicolau, João Oliveira et al.

No two people are alike. We usually ignore this diversity as we have the capability to adapt and, without noticing, become experts in interfaces that were probably misadjusted to begin with. This adaptation is not always at the user's reach. One neglected group is the blind. Spatial ability, memory, and tactile sensitivity are some characteristics that diverge between users. Regardless, all are presented with the same methods ignoring their capabilities and needs. Interaction with mobile devices is highly visually demanding which widens the gap between blind people. Our research goal is to identify the individual attributes that influence mobile interaction, considering the blind, and match them with mobile interaction modalities in a comprehensive and extensible design space. We aim to provide knowledge both for device design, device prescription and interface adaptation.

Tiago Guerreiro

Mobile phones pervade our daily lives and play ever expanding roles in many contexts. Their ubiquitousness makes them pivotal in empowering disabled people. However, if no inclusive approaches are provided, it becomes a strong vehicle of exclusion. Even though current solutions try to compensate for the lack of sight, not all information reaches the blind user. Good spatial ability is still required to make sense of the device and its interface, as well as the need to memorize positions on screen or keys and associated actions in a keypad. Those problems are compounded by many individual attributes such as age, age of blindness onset or tactile sensitivity which often are forgotten by designers. Worse, the entire blind population is recurrently thought of as homogeneous (often stereotypically so). Thus all users face the same solutions, ignoring their specific capabilities and needs. We usually ignore this diversity as we have the ability to adapt and become experts in interfaces that were probably maladjusted to begin with. This adaptation is not always within reach. Interaction with mobile devices is highly visually demanding which widens this gap amongst blind people. It is paramount to understand the impact of individual differences and their relationship with demands to enable the deployment of more inclusive solutions. We explore individual differences among blind people and assess how they are related with mobile interface demands, both at low (e.g. performing an on-screen gesture) and high level (text-entry) tasks. Results confirmed that different ability levels have significant impact on the performance attained by a blind person. Particularly, otherwise ignored attributes like tactile acuity, pressure sensitivity, spatial ability or verbal IQ have shown to be matched with specific mobile demands and parametrizations.

Hugo Nicolau, João Guerreiro, Tiago Guerreiro

Mobile devices gather the communication capabilities as no other gadget. Plus, they now comprise a wider set of applications while still maintaining reduced size and weight. They have started to include accessibility features that enable the inclusion of disabled people. However, these inclusive efforts still fall short considering the possibilities of such devices. This is mainly due to the lack of interoperability and extensibility of current mobile operating systems (OS). In this paper, we present a case study of a multi-impaired person where access to basic mobile applications was provided in an applicational basis. We outline the main flaws in current mobile OS and suggest how these could further empower developers to provide accessibility components. These could then be compounded to provide system-wide inclusion to a wider range of (multi)-impairments.