HCDec 10, 2025
ImageTalk: Designing a Multimodal AAC Text Generation System Driven by Image Recognition and Natural Language GenerationBoyin Yang, Puming Jiang, Per Ola Kristensson
People living with Motor Neuron Disease (plwMND) frequently encounter speech and motor impairments that necessitate a reliance on augmentative and alternative communication (AAC) systems. This paper tackles the main challenge that traditional symbol-based AAC systems offer a limited vocabulary, while text entry solutions tend to exhibit low communication rates. To help plwMND articulate their needs about the system efficiently and effectively, we iteratively design and develop a novel multimodal text generation system called ImageTalk through a tailored proxy-user-based and an end-user-based design phase. The system demonstrates pronounced keystroke savings of 95.6%, coupled with consistent performance and high user satisfaction. We distill three design guidelines for AI-assisted text generation systems design and outline four user requirement levels tailored for AAC purposes, guiding future research in this field.
6.1HCMar 27
Exploring a Design Framework for Children's Agency through Participatory DesignBoyin Yang, Jun Zhao
Children's agency plays a critical role in shaping children's autonomy, participation, and well-being in their interactions with digital systems, particularly in emerging child-AI contexts. However, how designers currently understand and reason about children's agency in practice remains underexplored. In this paper, we examine designers's engagement with children's agency through a participatory workshop in which we introduce a design-for-agency framework that supports designers externalising the consideration of agency in their design contexts. We find that while participants are committed to implementing ethical AI systems for children, they often struggle to understand why agency matters and how it can be operationalised in practice. Our agency design framework provided designers with a structured way to translate implicit, experience-based judgments into explicit articulation of agency trade-offs while acknowledging the associated design complexity. We conclude by offering initial insights into supporting designers' reasoning about children's agency and outlining directions for future research.