Jennifer Mankoff

Xuhai Xu, Han Zhang, Yasaman Sefidgar et al. · uw

Recent research has demonstrated the capability of behavior signals captured by smartphones and wearables for longitudinal behavior modeling. However, there is a lack of a comprehensive public dataset that serves as an open testbed for fair comparison among algorithms. Moreover, prior studies mainly evaluate algorithms using data from a single population within a short period, without measuring the cross-dataset generalizability of these algorithms. We present the first multi-year passive sensing datasets, containing over 700 user-years and 497 unique users' data collected from mobile and wearable sensors, together with a wide range of well-being metrics. Our datasets can support multiple cross-dataset evaluations of behavior modeling algorithms' generalizability across different users and years. As a starting point, we provide the benchmark results of 18 algorithms on the task of depression detection. Our results indicate that both prior depression detection algorithms and domain generalization techniques show potential but need further research to achieve adequate cross-dataset generalizability. We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.

Joshua Gluck, Christian Koehler, Jennifer Mankoff et al.

Heating, Ventilation, and Cooling (HVAC) systems are often the most significant contributor to the energy usage, and the operational cost, of large office buildings. Therefore, to understand the various factors affecting the energy usage, and to optimize the operational efficiency of building HVAC systems, energy analysts and architects often create simulations (e.g., EnergyPlus or DOE-2), of buildings prior to construction or renovation to determine energy savings and quantify the Return-on-Investment (ROI). While useful, these simulations usually use static HVAC control strategies such as lowering room temperature at night, or reactive control based on simulated room occupancy. Recently, advances have been made in HVAC control algorithms that predict room occupancy. However, these algorithms depend on costly sensor installations and the tradeoffs between predictive accuracy, energy savings, comfort and expenses are not well understood. Current simulation frameworks do not support easy analysis of these tradeoffs. Our contribution is a simulation framework that can be used to explore this design space by generating objective estimates of the energy savings and occupant comfort for different levels of HVAC prediction and control performance. We validate our framework on a real-world occupancy dataset spanning 6 months for 235 rooms in a large university office building. Using the gold standard of energy use modeling and simulation (Revit and Energy Plus), we compare the energy consumption and occupant comfort in 29 independent simulations that explore our parameter space. Our results highlight a number of potentially useful tradeoffs with respect to energy savings, comfort, and algorithmic performance among predictive, reactive, and static schedules, for a stakeholder of our building.

Jennifer Mankoff, Devva Kasnitz, Disability Studies et al.

This response to the RFI considers the potential for biometrics to help or harm disabled people2. Biometrics are already integrated into many aspects of daily life, from airport travel to mobile phone use. Yet many of these systems are not accessible to people who experience different kinds of disability exclusion . Different personal characteristics may impact any or all of the physical (DNA, fingerprints, face or retina) and behavioral (gesture, gait, voice) characteristics listed in the RFI as examples of biometric signals.

Tobias Weinberg, Aaleyah Lewis, Ricardo E. Gonzalez Penuela et al.

Voice is a central element of identity. We recognize people by their voice, and we uniquely express who we are with it. For people who rely on augmentative and alternative communication~(AAC) systems, such as speech-generating devices~(SGD), the device's voice becomes an identity marker others associate with them. Yet, it is hard to find a voice that truly aligns with one's identity both linguistically and culturally. Although modern AI-generated voices can reproduce diverse accents and speaking styles, AAC users still lack accessible ways to articulate how they want an identity-aligned voice to sound like. We first conducted a survey of AAC users (across eight countries) to characterize current voice representation, finding that non-binary, transgender, and non-US-born respondents rated their current voice support identity alignment consistently lower than other respondents. To examine how AAC users respond to voices designed to reflect their cultural identity, we built a tool that elicits cultural markers through guided questions and generates personalized voice candidates for participants to hear and reflect on. After participants heard the voices, we interviewed them to examine what it means for a voice to feel culturally representative, how they interpreted voices with cultural connotations, and how these voices shaped their sense of identity and agency. Our findings show that cultural voice alignment runs deeper than accent or language alone; it touches on belonging, self-recognition, and what it means to be heard as who you are.

Jennifer Mankoff, Janice Light, James Coughlan et al.

We argue that there is a need for Accessibility to be represented in several important domains: - Capitalize on the new capabilities AI provides - Support for open source development of AI, which can allow disabled and disability focused professionals to contribute, including - Development of Accessibility Apps which help realise the promise of AI in accessibility domains - Open Source Model Development and Validation to ensure that accessibility concerns are addressed in these algorithms - Data Augmentation to include accessibility in data sets used to train models - Accessible Interfaces that allow disabled people to use any AI app, and to validate its outputs - Dedicated Functionality and Libraries that can make it easy to integrate AI support into a variety of settings and apps. - Data security and privacy and privacy risks including data collected by AI based accessibility technologies; and the possibility of disability disclosure. - Disability-specific AI risks and biases including both direct bias (during AI use by the disabled person) and indirect bias (when AI is used by someone else on data relating to a disabled person).

Kelly Mack, Megan Hofmann, Udaya Lakshmi et al.

The NIH 3D Print Exchange is a public and open source repository for primarily 3D printable medical device designs with contributions from expert-amateur makers, engineers from industry and academia, and clinicians. In response to the COVID-19 pandemic, a collection was formed to foster submissions of low-cost, local manufacture of personal protective equipment (Personal Protective Equipment (PPE)). We systematically evaluated the 623 submissions in this collection to understand: what makers contributed, how they were made, who made them, and key characteristics of their designs. Our analysis reveals an immediate design convergence to derivatives of a few initial designs affiliated with NIH partners (e.g., universities, the Veteran's Health Administration, America Makes) and major for-profit groups (e.g., Prusa). The NIH worked to review safe and effective designs but was quickly overloaded by derivative works. We found that the vast majority were never reviewed (81.3%) while 10.4% of those reviewed were deemed safe for clinical (5.6%) or community use (4.8%). Our work contributes insights into: the outcomes of distributed, community-based, medical making; features the community accepted as "safe" making; and how platforms can support regulated maker activities in high-risk domains (e.g., healthcare).

Kate Glazko, Yusuf Mohammed, Ben Kosa et al.

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

Momona Yamagami, Kelly Mack, Jennifer Mankoff et al.

Many individuals with disabilities and/or chronic conditions (da/cc) experience symptoms that may require intermittent or on-going medical care. However, healthcare is an often-overlooked domain for accessibility work, where access needs associated with temporary and long-term disability must be addressed to increase the utility of physical and digital interactions with healthcare workers and spaces. Our work focuses on a specific domain of healthcare often used by individuals with da/cc: physical therapy (PT). Through a twelve-person interview study, we examined how people's access to PT for their da/cc is hampered by social (e.g., physically visiting a PT clinic) and physiological (e.g., chronic pain) barriers, and how technology could improve PT access. In-person PT is often inaccessible to our participants due to lack of transportation and insufficient insurance coverage. As such, many of our participants relied on at-home PT to manage their da/cc symptoms and work towards PT goals. Participants felt that PT barriers, such as having particularly bad symptoms or feeling short on time, could be addressed with well-designed technology that flexibly adapts to the person's dynamically changing needs while supporting their PT goals. We introduce core design principles (adaptability, movement tracking, community building) and tensions (insurance) to consider when developing technology to support PT access. Rethinking da/cc access to PT from a lens that includes social and physiological barriers presents opportunities to integrate accessibility and adaptability into PT technology.

Yasaman S. Sefidgar, Paula S. Nurius, Amanda Baughan et al.

Perceived discrimination is common and consequential. Yet, little support is available to ease handling of these experiences. Addressing this gap, we report on a need-finding study to guide us in identifying relevant technologies and their requirements. Specifically, we examined unfolding experiences of perceived discrimination among college students and found factors to address in providing meaningful support. We used semi-structured retrospective interviews with 14 students to understand their perceptions, emotions, and coping in response to discriminatory behaviors within the prior ten-week period. These 14 students were among 90 who provided experience sampling reports of unfair treatment over the same ten-week period. We found that discrimination is more distressing if students face related academic and social struggles or when the incident triggers beliefs of inefficacy. We additionally identified patterns of effective coping. By grounding the findings in an extended stress processing framework, we offer a principled approach to intervention design, which we illustrate through incident-specific and proactive intervention paradigms.

Kirstin Early, Jennifer Mankoff, Stephen E. Fienberg

Online surveys have the potential to support adaptive questions, where later questions depend on earlier responses. Past work has taken a rule-based approach, uniformly across all respondents. We envision a richer interpretation of adaptive questions, which we call dynamic question ordering (DQO), where question order is personalized. Such an approach could increase engagement, and therefore response rate, as well as imputation quality. We present a DQO framework to improve survey completion and imputation. In the general survey-taking setting, we want to maximize survey completion, and so we focus on ordering questions to engage the respondent and collect hopefully all information, or at least the information that most characterizes the respondent, for accurate imputations. In another scenario, our goal is to provide a personalized prediction. Since it is possible to give reasonable predictions with only a subset of questions, we are not concerned with motivating users to answer all questions. Instead, we want to order questions to get information that reduces prediction uncertainty, while not being too burdensome. We illustrate this framework with an example of providing energy estimates to prospective tenants. We also discuss DQO for national surveys and consider connections between our statistics-based question-ordering approach and cognitive survey methodology.

Julian Ramos, Zhen Li, Johana Rosas et al.

Pointing devices that reside on the keyboard can reduce the overall time needed to perform mixed pointing and typing tasks, since the hand of the user does not have to reach for the pointing device. However, previous implementations of this kind of device have a higher movement time compared to the mouse and trackpad due to large error rate, low speed and spatial resolution. In this paper we introduce Keyboard Surface Interaction (KSI), an interaction approach that turns the surface of a keyboard into an interaction surface and allows users to rest their hands on the keyboard at all times to minimize fatigue. We developed a proof-of-concept implementation, Fingers, which we optimized over a series of studies. Finally, we evaluated Fingers against the mouse and trackpad in a user study with 25 participants on a Fitts law test style, mixed typing and pointing task. Results showed that for users with more exposure to KSI, our KSI device had better performance (reduced movement and homing time) and reduced discomfort compared to the trackpad. When compared to the mouse, KSI had reduced homing time and reduced discomfort, but increased movement time. This interaction approach is not only a new way to capitalize on the space on top of the keyboard, but also a call to innovate and think beyond the touchscreen, touchpad, and mouse as our main pointing devices. The results of our studies serve as a specification for future KSI devices.