Taeahn Kwon

Taeahn Kwon, Minkyung Jeong, Eon-Suk Ko et al.

To acquire language, children need rich language input. However, many parents find it difficult to provide children with sufficient language input, which risks delaying their language development. To aid these parents, we design Captivate!, the first system that provides contextual language guidance to parents during play. Our system tracks both visual and spoken language cues to infer targets of joint attention, enabling the real-time suggestion of situation-relevant phrases for the parent. We design our system through a user-centered process with immigrant families--a highly vulnerable yet understudied population--as well as professional speech language therapists. Next, we evaluate Captivate! on parents with children aged 1-3 to observe improvements in responsive language use. We share insights into developing contextual guidance technology for linguistically diverse families.

Sitong Wang, Savvas Petridis, Taeahn Kwon et al.

Pop culture is an important aspect of communication. On social media people often post pop culture reference images that connect an event, product or other entity to a pop culture domain. Creating these images is a creative challenge that requires finding a conceptual connection between the users' topic and a pop culture domain. In cognitive theory, this task is called conceptual blending. We present a system called PopBlends that automatically suggests conceptual blends. The system explores three approaches that involve both traditional knowledge extraction methods and large language models. Our annotation study shows that all three methods provide connections with similar accuracy, but with very different characteristics. Our user study shows that people found twice as many blend suggestions as they did without the system, and with half the mental demand. We discuss the advantages of combining large language models with knowledge bases for supporting divergent and convergent thinking.