Justin Edwards

Helin Cihan, Yunhan Wu, Paola Peña et al.

Conversational User Interfaces such as Voice Assistants are hugely popular. Yet they are designed to be monolingual by default, lacking support for, or sensitivity to, the bilingual dialogue experience. In this provocation paper, we highlight the language production challenges faced in VA interaction for bilingual users. We argue that, by facilitating phenomena seen in bilingual interaction, such as code-switching, we can foster a more inclusive and improved user experience for bilingual users. We also explore ways that this might be achieved, through the support of multiple language recognition as well as being sensitive to the preferences of code-switching in speech output.

Justin Edwards, Orla Cooney, Rachel Edwards

Alexa Skills are used for a variety of daily routines and purposes, but little research has focused on a key part of many people's daily lives: their pets. We present a systematic review categorizing the purposes of 88 Alexa Skills aimed at pets and pet owners and introduce a veterinary perspective to assess their benefits and risks. We present 8 themes of the purposes for Skills aimed at pets and their owners: Calming, Animal Audience, Smart Device, Tracking, Training and Health, Translator, Entertainment/Trivia, and Other - Human Audience. Broadly, we find that these purposes mirror the purposes people have for using Alexa overall, and they largely are supported by veterinary evidence, though caution must be used when Skills relate to animal health. More collaboration between Conversational Agent researchers and animal scientists is called for to better understand the efficacy of using Alexa with pets.

Anna Bleakley, Daniel Rough, Justin Edwards et al.

Video conferencing systems have long facilitated work-related conversations among remote teams. However, social distancing due to the COVID-19 pandemic has forced colleagues to use video conferencing platforms to additionally fulfil social needs. Social talk, or informal talk, is an important workplace practice that is used to build and maintain bonds in everyday interactions among colleagues. Currently, there is a limited understanding of how video conferencing facilitates multiparty social interactions among colleagues. In our paper, we examine social talk practices during the COVID-19 pandemic among remote colleagues through semi-structured interviews. We uncovered three key themes in our interviews, discussing 1) the changing purposes and opportunities afforded by using video conferencing for social talk with colleagues, 2) how the nature of existing relationships and status of colleagues influences social conversations and 3) the challenges and changing conversational norms around politeness and etiquette when using video conferencing to hold social conversations. We discuss these results in relation to the impact that video conferencing tools have on remote social talk between colleagues and outline design and best practice considerations for multiparty videoconferencing social talk in the workplace.

Justin Edwards, Christian Janssen, Sandy Gould et al.

Current speech agent interactions are typically user-initiated, limiting the interactions they can deliver. Future functionality will require agents to be proactive, sometimes interrupting users. Little is known about how these spoken interruptions should be designed, especially in urgent interruption contexts. We look to inform design of proactive agent interruptions through investigating how people interrupt others engaged in complex tasks. We therefore developed a new technique to elicit human spoken interruptions of people engaged in other tasks. We found that people interrupted sooner when interruptions were urgent. Some participants used access rituals to forewarn interruptions, but most rarely used them. People balanced speed and accuracy in timing interruptions, often using cues from the task they interrupted. People also varied phrasing and delivery of interruptions to reflect urgency. We discuss how our findings can inform speech agent design and how our paradigm can help gain insight into human interruptions in new contexts.

Justin Edwards, Allison Perrone, Philip R. Doyle

Language models and conversational systems are growing increasingly advanced, creating outputs that may be mistaken for humans. Consumers may thus be misled by advertising, media reports, or vagueness regarding the role of automation in the production of language. We propose a taxonomy of language automation, based on the SAE levels of driving automation, to establish a shared set of terms for describing automated language. It is our hope that the proposed taxonomy can increase transparency in this rapidly advancing field.

Benjamin R. Cowan, Philip Doyle, Justin Edwards et al.

The assumptions we make about a dialogue partner's knowledge and communicative ability (i.e. our partner models) can influence our language choices. Although similar processes may operate in human-machine dialogue, the role of design in shaping these models, and their subsequent effects on interaction are not clearly understood. Focusing on synthesis design, we conduct a referential communication experiment to identify the impact of accented speech on lexical choice. In particular, we focus on whether accented speech may encourage the use of lexical alternatives that are relevant to a partner's accent, and how this is may vary when in dialogue with a human or machine. We find that people are more likely to use American English terms when speaking with a US accented partner than an Irish accented partner in both human and machine conditions. This lends support to the proposal that synthesis design can influence partner perception of lexical knowledge, which in turn guide user's lexical choices. We discuss the findings with relation to the nature and dynamics of partner models in human machine dialogue.

Justin Edwards, He Liu, Tianyu Zhou et al.

Intelligent personal assistants (IPAs) are supposed to help us multitask. Yet the impact of IPA use on multitasking is not clearly quantified, particularly in situations where primary tasks are also language based. Using a dual task paradigm, our study observes how IPA interactions impact two different types of writing primary tasks; copying and generating content. We found writing tasks that involve content generation, which are more cognitively demanding and share more of the resources needed for IPA use, are significantly more disrupted by IPA interaction than less demanding tasks such as copying content. We discuss how theories of cognitive resources, including multiple resource theory and working memory, explain these results. We also outline the need for future work how interruption length and relevance may impact primary task performance as well as the need to identify effects of interruption timing in user and IPA led interruptions.

Allison Perrone, Justin Edwards

Chatbots are popular for both task-oriented conversations and unstructured conversations with web users. Several different approaches to creating comedy and art exist across the field of computational creativity. Despite the popularity and ease of use of chatbots, there have not been any attempts by artists or comedians to use these systems for comedy performances. We present two initial attempts to do so from our comedy podcast and call for future work toward both designing chatbots for performance and for performing alongside chatbots.

Leigh Clark, Nadia Pantidi, Orla Cooney et al.

Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in supporting long-term human-agent relationships, it is paramount that HCI focuses efforts on delivering this promise. We aim to understand what people value in conversation and how this should manifest in agents. Findings from a series of semi-structured interviews show people make a clear dichotomy between social and functional roles of conversation, emphasising the long-term dynamics of bond and trust along with the importance of context and relationship stage in the types of conversations they have. People fundamentally questioned the need for bond and common ground in agent communication, shifting to more utilitarian definitions of conversational qualities. Drawing on these findings we discuss key challenges for conversational agent design, most notably the need to redefine the design parameters for conversational agent interaction.

Leigh Clark, Phillip Doyle, Diego Garaialde et al.

Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology \& accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.