Andrew Johnson

Abhinav Kumar, Barbara Di Eugenio, Abari Bhattacharya et al.

Reference resolution, which aims to identify entities being referred to by a speaker, is more complex in real world settings: new referents may be created by processes the agents engage in and/or be salient only because they belong to the shared physical setting. Our focus is on resolving references to visualizations on a large screen display in multimodal dialogue; crucially, reference resolution is directly involved in the process of creating new visualizations. We describe our annotations for user references to visualizations appearing on a large screen via language and hand gesture and also new entity establishment, which results from executing the user request to create a new visualization. We also describe our reference resolution pipeline which relies on an information-state architecture to maintain dialogue context. We report results on detecting and resolving references, effectiveness of contextual information on the model, and under-specified requests for creating visualizations. We also experiment with conventional CRF and deep learning / transformer models (BiLSTM-CRF and BERT-CRF) for tagging references in user utterance text. Our results show that transfer learning significantly boost performance of the deep learning methods, although CRF still out-performs them, suggesting that conventional methods may generalize better for low resource data.

Yalda Ghasemi, Ankit Singh, Myunghee Kim et al.

An augmented reality (AR) environment includes a set of digital elements with which the users interact while performing certain tasks. Recent AR head-mounted displays allow users to select how these elements are presented. However, few studies have been conducted to examine the effect of the way of presenting augmented content on user performance and workload. This study aims to evaluate two methods of presenting augmented content - world-locked and head-locked modes in a data entry task. A total of eighteen participants performed the data entry task in this study. The effectiveness of each mode is evaluated in terms of task performance, muscle activity, perceived workload, and usability. The results show that the task completion time is shorter and the typing speed is significantly faster in the head-locked mode while the world-locked mode achieved higher scores in terms of preference. The findings of this study can be applied to AR user interfaces to improve content presentation and enhance the user experience.

Ryan Budahazy, Lu Cheng, Yihuan Huang et al.

The California Innocence Project (CIP), a clinical law school program aiming to free wrongfully convicted prisoners, evaluates thousands of mails containing new requests for assistance and corresponding case files. Processing and interpreting this large amount of information presents a significant challenge for CIP officials, which can be successfully aided by topic modeling techniques.In this paper, we apply Non-negative Matrix Factorization (NMF) method and implement various offshoots of it to the important and previously unstudied data set compiled by CIP. We identify underlying topics of existing case files and classify request files by crime type and case status (decision type). The results uncover the semantic structure of current case files and can provide CIP officials with a general understanding of newly received case files before further examinations. We also provide an exposition of popular variants of NMF with their experimental results and discuss the benefits and drawbacks of each variant through the real-world application.