Anthony Tomasic

Oscar J. Romero, John Zimmerman, Aaron Steinfeld et al.

This paper explores the integration of two AI subdisciplines employed in the development of artificial agents that exhibit intelligent behavior: Large Language Models (LLMs) and Cognitive Architectures (CAs). We present three integration approaches, each grounded in theoretical models and supported by preliminary empirical evidence. The modular approach, which introduces four models with varying degrees of integration, makes use of chain-of-thought prompting, and draws inspiration from augmented LLMs, the Common Model of Cognition, and the simulation theory of cognition. The agency approach, motivated by the Society of Mind theory and the LIDA cognitive architecture, proposes the formation of agent collections that interact at micro and macro cognitive levels, driven by either LLMs or symbolic components. The neuro-symbolic approach, which takes inspiration from the CLARION cognitive architecture, proposes a model where bottom-up learning extracts symbolic representations from an LLM layer and top-down guidance utilizes symbolic representations to direct prompt engineering in the LLM layer. These approaches aim to harness the strengths of both LLMs and CAs, while mitigating their weaknesses, thereby advancing the development of more robust AI systems. We discuss the tradeoffs and challenges associated with each approach.

Alex Haig, Shirley Anugrah Hayati, Anthony Tomasic

Automated passenger counting (APC) technology is central to many aspects of the public transit experience. APC information informs public transit planners about utilization in a public transit system and operations about dynamic fluctuations in demand. Perhaps most importantly, APC information provides one metric to the rider experience - standing during a long ride because of a crowded vehicle is an unpleasant experience. Several technologies have been successfully used for APC including light beam sensing and video image analysis. However, these technologies are expensive and must be installed in buses. In this paper, we analyze a new source of data using statistical models: rider smartphone accelerometers. Smartphones are ubiquitous in society and accelerometers have been shown to accurately model user states such as walking and sitting. We extend these models to use accelerometers to detect if the rider is standing or sitting on a bus. Standing riders are a signal that the bus is crowded. This paper provides evidence that user smartphones are a valid source of participatory sensing and thus a new source of automated passenger counting data.

Mihir Kale, Aditya Siddhant, Sreyashi Nag et al.

Pre-trained word embeddings are the primary method for transfer learning in several Natural Language Processing (NLP) tasks. Recent works have focused on using unsupervised techniques such as language modeling to obtain these embeddings. In contrast, this work focuses on extracting representations from multiple pre-trained supervised models, which enriches word embeddings with task and domain specific knowledge. Experiments performed in cross-task, cross-domain and cross-lingual settings indicate that such supervised embeddings are helpful, especially in the low-resource setting, but the extent of gains is dependent on the nature of the task and domain. We make our code publicly available.

Aaron Steinfeld, John Zimmerman, Anthony Tomasic

There is great promise in creating effective technology experiences during situationally-induced impairments and disabilities through the combination of universal design and adaptive interfaces. We believe this combination is a powerful approach for meeting the UX needs of people with disabilities, including those which are temporary in nature. Research in each of these areas, and the combination, illustrates this promise.

Bhavya Karki, Fan Hu, Nithin Haridas et al.

In this paper, we describe a dataset and baseline result for a question answering that utilizes web tables. It contains commonly asked questions on the web and their corresponding answers found in tables on websites. Our dataset is novel in that every question is paired with a table of a different signature. In particular, the dataset contains two classes of tables: entity-instance tables and the key-value tables. Each QA instance comprises a table of either kind, a natural language question, and a corresponding structured SQL query. We build our model by dividing question answering into several tasks, including table retrieval and question element classification, and conduct experiments to measure the performance of each task. We extract various features specific to each task and compose a full pipeline which constructs the SQL query from its parts. Our work provides qualitative results and error analysis for each task, and identifies in detail the reasoning required to generate SQL expressions from natural language questions. This analysis of reasoning informs future models based on neural machine learning.

Shirley Anugrah Hayati, Raphael Olivier, Pravalika Avvaru et al.

In models to generate program source code from natural language, representing this code in a tree structure has been a common approach. However, existing methods often fail to generate complex code correctly due to a lack of ability to memorize large and complex structures. We introduce ReCode, a method based on subtree retrieval that makes it possible to explicitly reference existing code examples within a neural code generation model. First, we retrieve sentences that are similar to input sentences using a dynamic-programming-based sentence similarity scoring method. Next, we extract n-grams of action sequences that build the associated abstract syntax tree. Finally, we increase the probability of actions that cause the retrieved n-gram action subtree to be in the predicted code. We show that our approach improves the performance on two code generation tasks by up to +2.6 BLEU.