SweeperBot: Making 3D Browsing Accessible through View Analysis and Visual Question Answering
This addresses accessibility challenges for blind and low-vision users in 3D browsing, representing an incremental improvement by integrating existing techniques into a new system.
The paper tackles the problem of making 3D models accessible to screen reader users by introducing SweeperBot, a system that uses visual question answering to help users explore and compare models, with feasibility demonstrated in an expert review with 10 BLV users and description quality validated in a survey with 30 sighted participants.
Accessing 3D models remains challenging for Screen Reader (SR) users. While some existing 3D viewers allow creators to provide alternative text, they often lack sufficient detail about the 3D models. Grounded on a formative study, this paper introduces SweeperBot, a system that enables SR users to leverage visual question answering to explore and compare 3D models. SweeperBot answers SR users' visual questions by combining an optimal view selection technique with the strength of generative- and recognition-based foundation models. An expert review with 10 Blind and Low-Vision (BLV) users with SR experience demonstrated the feasibility of using SweeperBot to assist BLV users in exploring and comparing 3D models. The quality of the descriptions generated by SweeperBot was validated by a second survey study with 30 sighted participants.