SimplerVoice: A Key Message & Visual Description Generator System for Illiteracy
This addresses the challenge of information access for low-literate individuals, though it appears incremental as it builds on existing text-to-visual generation methods for a specific domain.
The paper tackled the problem of helping low-literate adults navigate information by developing SimplerVoice, a system that generates key messages and visual descriptions for objects, achieving a performance score of 4.82 out of 5 in a user study on grocery product manuals.
We introduce SimplerVoice: a key message and visual description generator system to help low-literate adults navigate the information-dense world with confidence, on their own. SimplerVoice can automatically generate sensible sentences describing an unknown object, extract semantic meanings of the object usage in the form of a query string, then, represent the string as multiple types of visual guidance (pictures, pictographs, etc.). We demonstrate SimplerVoice system in a case study of generating grocery products' manuals through a mobile application. To evaluate, we conducted a user study on SimplerVoice's generated description in comparison to the information interpreted by users from other methods: the original product package and search engines' top result, in which SimplerVoice achieved the highest performance score: 4.82 on 5-point mean opinion score scale. Our result shows that SimplerVoice is able to provide low-literate end-users with simple yet informative components to help them understand how to use the grocery products, and that the system may potentially provide benefits in other real-world use cases