Jonathan Smith

Lily Hong Zhang, Smitha Milli, Karen Jusko et al.

How can large language models (LLMs) serve users with varying preferences that may conflict across cultural, political, or other dimensions? To advance this challenge, this paper establishes four key results. First, we demonstrate, through a large-scale multilingual human study with representative samples from five countries (N=15,000), that humans exhibit significantly more variation in preferences than the responses of 21 state-of-the-art LLMs. Second, we show that existing methods for preference dataset collection are insufficient for learning the diversity of human preferences even along two of the most salient dimensions of variability in global values, due to the underlying homogeneity of candidate responses. Third, we argue that this motivates the need for negatively-correlated sampling when generating candidate sets, and we show that simple prompt-based techniques for doing so significantly enhance the performance of alignment methods in learning heterogeneous preferences. Fourth, based on this novel candidate sampling approach, we collect and open-source Community Alignment, the largest and most representative multilingual and multi-turn preference dataset to date, featuring almost 200,000 comparisons from annotators spanning five countries. We hope that the Community Alignment dataset will be a valuable resource for improving the effectiveness of LLMs for a diverse global population.

Edward C. C. Steele, Dinesh Mane, Emilio Monti et al.

Despite the promising capability of multimodal foundation models, their application to the generation of meteorological products and services remains nascent. To accelerate aspiration and adoption, we explore the novel use of a vision language model for writing the iconic Shipping Forecast text directly from video-encoded gridded weather data. These early results demonstrate promising scalable technological opportunities for enhancing production efficiency and service innovation within the weather enterprise and beyond.

Kin Kwan Leung, Clayton Rooke, Jonathan Smith et al.

Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues. Unlike existing feature removal explanation methods, WinIT explicitly accounts for the temporal dependence between different observations of the same feature in the construction of its importance score. Furthermore, WinIT captures the varying importance of a feature over time, by summarizing its importance over a window of past time steps. We conduct an extensive empirical study on synthetic and real-world data, compare against a wide range of leading explainability methods, and explore the impact of various evaluation strategies. Our results show that WinIT achieves significant gains over existing methods, with more consistent performance across different evaluation metrics. The code for our work is publicly available at \url{https://github.com/layer6ai-labs/WinIT}.

Kadijatou Diallo, Jonathan Smith, Chinasa T. Okolo et al.

Artificial Intelligence (AI) requires new ways of evaluating national technology use and strategy for African nations. We conduct a survey of existing 'readiness' assessments both for general digital adoption and for AI policy in particular. We conclude that existing global readiness assessments do not fully capture African states' progress in AI readiness and lay the groundwork for how assessments can be better used for the African context. We consider the extent to which these indicators map to the African context and what these indicators miss in capturing African states' on-the-ground work in meeting AI capability. Through case studies of four African nations of diverse geographic and economic dimensions, we identify nuances missed by global assessments and offer high-level policy considerations for how states can best improve their AI readiness standards and prepare their societies to capture the benefits of AI.

Jonathan Smith, Borna Ghotbi, Seungeun Yi et al.

We consider the task of discovering categories of non-pharmaceutical interventions during the evolving COVID-19 pandemic. We explore topic modeling on two corpora with national and international scope. These models discover existing categories when compared with human intervention labels while reduced human effort needed.

Anja Petri, Christian Schindler, Wolfgang Slany et al.

The constructionist approach is more interested in constructing personal experience than about acquiring information. It states that learning is most effective when building knowledge through active engagement. Experiential and discovery learning by challenges inspire creativity, and projects allow independent thinking and new ways of learning information. This paper describes how the "No One Left Behind" (NOLB) project plans to integrate this approach into school curricula using two concepts. The first one is to enable students to create their own games with Pocket Code by using its easy-to-learn visual programming language. The second concept is to foster collaboration and teamwork through hands-on sessions by conducting Game Jams using Pocket Code, so called Pocket Game Jams. We present insights into such a Pocket Game Jam and give an outlook on how we will use this concept.

Bernadette Spieler, Anja Petri, Christian Schindler et al.

Game jams are a way to create games under fast-paced conditions and certain constraints. The increase in game jam events all over the world, their engaging and creative nature, with the aim of sharing results among players can be seen in the high participation rate of such events (2013: 16,705 participants from 319 jam sites in 63 countries produced 3248 games) . This promising concept can be easily transferred to a classroom setting.

Nelson Nauata, Jonathan Smith, Greg Mori

Videos are a rich source of high-dimensional structured data, with a wide range of interacting components at varying levels of granularity. In order to improve understanding of unconstrained internet videos, it is important to consider the role of labels at separate levels of abstraction. In this paper, we consider the use of the Bidirectional Inference Neural Network (BINN) for performing graph-based inference in label space for the task of video classification. We take advantage of the inherent hierarchy between labels at increasing granularity. The BINN is evaluated on the first and second release of the YouTube-8M large scale multilabel video dataset. Our results demonstrate the effectiveness of BINN, achieving significant improvements against baseline models.