Koji Yatani

Eden Shaveet, Zefan Sramek, Yumi Hamamoto et al.

Objective: Reliable identification of pro-eating disorder (pro-ED) content online suffers from two pervasive problems: 1) existing methods predominantly rely on text-based signals, failing to capture the inherently multimodal nature of multimedia content; and 2) these methods struggle to keep pace with the rapid evolution of references, memes, terminology, and contextual cues that underlie this content. Together, these limitations point to a gap: the absence of an expert-annotated reference standard capable of supporting real-time research and robust multimodal detection model training for pro-ED content on short-form video platforms. Method: To address this, we propose "zeitgeist-aware" multimodal (ZAM) datasets: continuously curated collections of annotated multimodal pro-ED content with inclusion criteria that evolve alongside the memetic zeitgeist: the variable essence of what is considered pro-ED as new media and references come into the cultural zeitgeist and are absorbed and interpreted in online spaces. Results: We present a rationale for such datasets, define their core characteristics, outline approaches for their curation, and describe our progress toward that end. Discussion: This dataset and pipeline architecture may benefit researchers across several fields who are interested in how pro-ED sentiment is encoded and transmitted through short-form video content across time, including for the purpose of responsive moderation efforts.

Shitao Fang, Koji Yatani, Kasper Hornbæk

In HCI, frameworks function as a type of theoretical contribution, often supporting ideation, design, and evaluation. Yet, little is known about how they are actually used, what functions they serve, and which scholarly practices that shape them. To address this gap, we conducted a systematic review of 615 papers from a decade of CHI proceedings (2015-2024) that prominently featured the term framework. We classified these papers into six engagement types. We then examined the role, form, and essential components of newly proposed frameworks through a functional typology, analyzing how they are constructed, validated, and articulated for reuse. Our results show that enthusiasm for proposing new frameworks exceeds the willingness to iterate on existing ones. They also highlight the ambiguity in the function of frameworks and the scarcity of systematic validation. Based on these insights, we call for more rigorous, reflective, and cumulative practices in the development and use of frameworks in HCI.

Quan M. Tran, Zhuo Huang, Wenbin Zhang et al.

Autonomous agents excel in self-improvement through reflection and iterative refinement, which reuse successful task trajectories as in-context examples to assist subsequent reasoning. However, shifting across tasks often introduces a context mismatch. Hence, existing approaches either discard the trajectories or manipulate them using heuristics, leading to a non-negligible fine-tuning cost or unguaranteed performance. To bridge this gap, we reveal a context-trajectory correlation, where shifts of context are highly parallel with shifts of trajectory. Based on this finding, we propose BrIdge contextual gap FoR imprOvised trajectory STeering (Bifrost), a training-free method that leverages context differences to precisely guide the adaptation of previously solved trajectories towards the target task, mitigating the misalignment caused by context shifts. Our trajectory adaptation is conducted at the representation level using agent hidden states, ensuring trajectory transformation accurately aligns with the target context in a shared space. Across diverse benchmarks, Bifrost consistently outperforms existing trajectory reuse and finetuned self-improvement methods, demonstrating that agents can effectively leverage past experiences despite substantial context shifts.

Kavindu Ravishan, Dániel Szabó, Niels van Berkel et al.

Online reviews help people make better decisions. Review platforms usually depend on typed input, where leaving a good review requires significant effort because users must carefully organize and articulate their thoughts. This may discourage users from leaving comprehensive and high-quality reviews, especially when they are on the go. To address this challenge, we developed Vocalizer, a mobile application that enables users to provide reviews through voice input, with enhancements from a large language model (LLM). In a longitudinal study, we analysed user interactions with the app, focusing on AI-driven features that help refine and improve reviews. Our findings show that users frequently utilized the AI agent to add more detailed information to their reviews. We also show how interactive AI features can improve users self-efficacy and willingness to share reviews online. Finally, we discuss the opportunities and challenges of integrating AI assistance into review-writing systems.

Zhongyi Zhou, Koji Yatani

Interactive Machine Teaching systems allow users to create customized machine learning models through an iterative process of user-guided training and model assessment. They primarily offer confidence scores of each label or class as feedback for assessment by users. However, we observe that such feedback does not necessarily suffice for users to confirm the behavior of the model. In particular, confidence scores do not always offer the full understanding of what features in the data are used for learning, potentially leading to the creation of an incorrectly-trained model. In this demonstration paper, we present a vision-based interactive machine teaching interface with real-time saliency map visualization in the assessment phase. This visualization can offer feedback on which regions of each image frame the current model utilizes for classification, thus better guiding users to correct the corresponding concepts in the iterative teaching.

Zhongyi Zhou, Anran Xu, Koji Yatani

The beauty of synchronized dancing lies in the synchronization of body movements among multiple dancers. While dancers utilize camera recordings for their practice, standard video interfaces do not efficiently support their activities of identifying segments where they are not well synchronized. This thus fails to close a tight loop of an iterative practice process (i.e., capturing a practice, reviewing the video, and practicing again). We present SyncUp, a system that provides multiple interactive visualizations to support the practice of synchronized dancing and liberate users from manual inspection of recorded practice videos. By analyzing videos uploaded by users, SyncUp quantifies two aspects of synchronization in dancing: pose similarity among multiple dancers and temporal alignment of their movements. The system then highlights which body parts and which portions of the dance routine require further practice to achieve better synchronization. The results of our system evaluations show that our pose similarity estimation and temporal alignment predictions were correlated well with human ratings. Participants in our qualitative user evaluation expressed the benefits and its potential use of SyncUp, confirming that it would enable quick iterative practice.

Ryo Suzuki, Jun Kato, Koji Yatani

Programming education is becoming important as demands on computer literacy and coding skills are growing. Despite the increasing popularity of interactive online learning systems, many programming courses in schools have not changed their teaching format from the conventional classroom setting. We see two research opportunities here. Students may have diverse expertise and experience in programming. Thus, particular content and teaching speed can be disengaging for experienced students or discouraging for novice learners. In a large classroom, instructors cannot oversee the learning progress of each student, and have difficulty matching teaching materials with the comprehension level of individual students. We present ClassCode, a web-based environment tailored to programming education in classrooms. Students can take online tutorials prepared by instructors at their own pace. They can then deepen their understandings by performing interactive coding exercises interleaved within tutorials. ClassCode tracks all interactions by each student, and summarizes them to instructors. This serves as a progress report, facilitating the instructors to provide additional explanations in-situ or revise course materials. Our user evaluation through a small lecture and expert review by instructors and teaching assistants confirm the potential of ClassCode by uncovering how it could address issues in existing programming courses at universities.

Ryo Suzuki, Koji Yatani, Mark D. Gross et al.

This paper presents Tabby, an interactive and explorable design tool for 3D printing textures. Tabby allows texture design with direct manipulation in the following workflow: 1) select a target surface, 2) sketch and manipulate a texture with 2D drawings, and then 3) generate 3D printing textures onto an arbitrary curved surface. To enable efficient texture creation, Tabby leverages an auto-completion approach which automates the tedious, repetitive process of applying texture, while allowing flexible customization. Our user evaluation study with seven participants confirms that Tabby can effectively support the design exploration of different patterns for both novice and experienced users.

Haojian Jin, Yale Song, Koji Yatani

Video consumption is being shifted from sit-and-watch to selective skimming. Existing video player interfaces, however, only provide indirect manipulation to support this emerging behavior. Video summarization alleviates this issue to some extent, shortening a video based on the desired length of a summary as an input variable. But an optimal length of a summarized video is often not available in advance. Moreover, the user cannot edit the summary once it is produced, limiting its practical applications. We argue that video summarization should be an interactive, mixed-initiative process in which users have control over the summarization procedure while algorithms help users achieve their goal via video understanding. In this paper, we introduce ElasticPlay, a mixed-initiative approach that combines an advanced video summarization technique with direct interface manipulation to help users control the video summarization process. Users can specify a time budget for the remaining content while watching a video; our system then immediately updates the playback plan using our proposed cut-and-forward algorithm, determining which parts to skip or to fast-forward. This interactive process allows users to fine-tune the summarization result with immediate feedback. We show that our system outperforms existing video summarization techniques on the TVSum50 dataset. We also report two lab studies (22 participants) and a Mechanical Turk deployment study (60 participants), and show that the participants responded favorably to ElasticPlay.

Ryo Suzuki, Tom Yeh, Koji Yatani et al.

Texture is an essential property of physical objects that affects aesthetics, usability, and functionality. However, designing and applying textures to 3D objects with existing tools remains difficult and time-consuming; it requires proficient 3D modeling skills. To address this, we investigated an auto-completion approach for efficient texture creation that automates the tedious, repetitive process of applying texture while allowing flexible customization. We developed techniques for users to select a target surface, sketch and manipulate a texture with 2D drawings, and then generate 3D printable textures onto an arbitrary curved surface. In a controlled experiment our tool sped texture creation by 80% over conventional tools, a performance gain that is higher with more complex target surfaces. This result confirms that auto-completion is powerful for creating 3D textures.