Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People
This addresses accessibility for blind and low vision people by evaluating AI tools, but it is incremental as it builds on prior research on human-assisted apps.
The study investigated how blind and low vision people use AI-powered scene description apps, finding common use cases like identifying objects and surprising ones like avoiding dangers, but users rated satisfaction and trust in the descriptions low (2.76/5 and 2.43/4).
"Scene description" applications that describe visual content in a photo are useful daily tools for blind and low vision (BLV) people. Researchers have studied their use, but they have only explored those that leverage remote sighted assistants; little is known about applications that use AI to generate their descriptions. Thus, to investigate their use cases, we conducted a two-week diary study where 16 BLV participants used an AI-powered scene description application we designed. Through their diary entries and follow-up interviews, users shared their information goals and assessments of the visual descriptions they received. We analyzed the entries and found frequent use cases, such as identifying visual features of known objects, and surprising ones, such as avoiding contact with dangerous objects. We also found users scored the descriptions relatively low on average, 2.76 out of 5 (SD=1.49) for satisfaction and 2.43 out of 4 (SD=1.16) for trust, showing that descriptions still need significant improvements to deliver satisfying and trustworthy experiences. We discuss future opportunities for AI as it becomes a more powerful accessibility tool for BLV users.