David Brown

Nathalie Riche, Anna Offenwanger, Frederic Gmeiner et al.

Chat-based prompts respond with verbose linear-sequential texts, making it difficult to explore and refine ambiguous intents, back up and reinterpret, or shift directions in creative AI-assisted design work. AI-Instruments instead embody "prompts" as interface objects via three key principles: (1) Reification of user-intent as reusable direct-manipulation instruments; (2) Reflection of multiple interpretations of ambiguous user-intents (Reflection-in-intent) as well as the range of AI-model responses (Reflection-in-response) to inform design "moves" towards a desired result; and (3) Grounding to instantiate an instrument from an example, result, or extrapolation directly from another instrument. Further, AI-Instruments leverage LLM's to suggest, vary, and refine new instruments, enabling a system that goes beyond hard-coded functionality by generating its own instrumental controls from content. We demonstrate four technology probes, applied to image generation, and qualitative insights from twelve participants, showing how AI-Instruments address challenges of intent formulation, steering via direct manipulation, and non-linear iterative workflows to reflect and resolve ambiguous intents.

Frederic Gmeiner, Nicolai Marquardt, Michael Bentley et al.

Despite Generative AI (GenAI) systems' potential for enhancing content creation, users often struggle to effectively integrate GenAI into their creative workflows. Core challenges include misalignment of AI-generated content with user intentions (intent elicitation and alignment), user uncertainty around how to best communicate their intents to the AI system (prompt formulation), and insufficient flexibility of AI systems to support diverse creative workflows (workflow flexibility). Motivated by these challenges, we created IntentTagger: a system for slide creation based on the notion of Intent Tags - small, atomic conceptual units that encapsulate user intent - for exploring granular and non-linear micro-prompting interactions for Human-GenAI co-creation workflows. Our user study with 12 participants provides insights into the value of flexibly expressing intent across varying levels of ambiguity, meta-intent elicitation, and the benefits and challenges of intent tag-driven workflows. We conclude by discussing the broader implications of our findings and design considerations for GenAI-supported content creation workflows.

Andrew Tran, Chris Bowes, David Brown et al.

Word sense disambiguation (WSD) is one of the main challenges in Computational Linguistics. TreeMatch is a WSD system originally developed using data from SemEval 2007 Task 7 (Coarse-grained English All-words Task) that has been adapted for use in SemEval 2010 Task 17 (All-words Word Sense Disambiguation on a Specific Domain). The system is based on a fully unsupervised method using dependency knowledge drawn from a domain specific knowledge base that was built for this task. When evaluated on the task, the system precision performs above the Most Frequent Selection baseline.

James V. Roggeveen, Erik Y. Wang, Will Flintoft et al.

Large language models (LLMs) have shown remarkable progress in mathematical problem-solving, but evaluation has largely focused on problems that have exact analytical solutions or involve formal proofs, often overlooking approximation-based problems ubiquitous in applied science and engineering. To fill this gap, we build on prior work and present HARDMath2, a dataset of 211 original problems covering the core topics in an introductory graduate applied math class, including boundary-layer analysis, WKB methods, asymptotic solutions of nonlinear partial differential equations, and the asymptotics of oscillatory integrals. This dataset was designed and verified by the students and instructors of a core graduate applied mathematics course at Harvard. We build the dataset through a novel collaborative environment that challenges students to write and refine difficult problems consistent with the class syllabus, peer-validate solutions, test different models, and automatically check LLM-generated solutions against their own answers and numerical ground truths. Evaluation results show that leading frontier models still struggle with many of the problems in the dataset, highlighting a gap in the mathematical reasoning skills of current LLMs. Importantly, students identified strategies to create increasingly difficult problems by interacting with the models and exploiting common failure modes. This back-and-forth with the models not only resulted in a richer and more challenging benchmark but also led to qualitative improvements in the students' understanding of the course material, which is increasingly important as we enter an age where state-of-the-art language models can solve many challenging problems across a wide domain of fields.

Andrew Edmonds, David Brown, Luciano Vinas et al.

We describe the use of machine learning algorithms to select high-quality measurements for the Mu2e experiment. This technique is important for experiments with backgrounds that arise due to measurement errors. The algorithms use multiple pieces of ancillary information that are sensitive to measurement quality to separate high-quality and low-quality measurements.

Kieran Woodward, Eiman Kanjo, David Brown

Mental well-being technologies possess many qualities that give them the potential to help people receive assessment and treatment who may otherwise not receive help due to fear of stigma or lack of resources. The combination of advances in sensors, microcontrollers and machine learning is leading to the emergence of dedicated tangible interfaces to monitor and promote positive mental well-being. However, there are key technical, ergonomic and aesthetic challenges to be overcome in order to make these interfaces effective and respond to users' needs. In this paper, the barriers to develop mental well-being tangible interfaces are discussed by identifying and examining the recent technological challenges machine learning, sensors, microcontrollers and batteries create.User-oriented challenges that face the development of mental well-being technologies are then considered ranging from user engagement during co-design and trials to ethical and privacy concerns.

Bernadette Spieler, Christian Schindler, Wolfgang Slany et al.

Game creation challenges in schools potentially provide engaging, goal-oriented, and interactive experiences in classes; thereby supporting the transfer of knowledge for learning in a fun and pedagogic manner. A key element of the ongoing European project No One Left Behind (NOLB) is to integrate a game-making teaching framework (GMTF) into the educational app Pocket Code. Pocket Code allows learners to create programs in a visual Lego-style way to facilitate learning how to code at secondary high schools. The concept of the NOLB GMTF is based on principles of the Universal Design for Learning (UDL) model. This framework provides a coherent approach to learning and teaching by integrating leisure oriented gaming methods into multi-discipline curricula. One output of this framework is the integration of game-based methods via game templates that refer to didactical scenarios that include a refined set of genres, assets, rules, challenges, and strategies. These templates allows: 1) teachers to start with a well-structured program, and 2) pupils to add content and adjust the code to integrate their own ideas. During the project game genres such as adventure, action, and quiz, as well as rewards or victory point mechanisms, have been embedded into different subjects, e.g., science, mathematics, and arts. The insights gained during the class hours were used to generate 13 game templates, which are integrated in Create@School (a new version of the Pocket Code app which targets schools). To test the efficiency of these templates, user experience (UX) tests were conducted during classes to compare games created by pupils who used templates and those who started to create a game from scratch. Preliminary results showed that these templates allow learners to focus on subject-relevant problem solving activities rather than on understanding the functionality of the app.