Feasibility of Interactive 3D Map for Remote Sighted Assistance
This work addresses challenges in assistive technology for people with vision impairments, though it is incremental as it builds on prior suggestions for using 3D reconstruction in RSA.
The paper tackled the problem of remote sighted assistance (RSA) by developing an interactive 3D map prototype to address agents' lack of environmental knowledge and orientation difficulties, resulting in significantly faster navigational assistance and reduced mental workload for agents in a study with thirteen agents and one user.
Remote sighted assistance (RSA) has emerged as a conversational assistive technology, where remote sighted workers, i.e., agents, provide real-time assistance to users with vision impairments via video-chat-like communication. Researchers found that agents' lack of environmental knowledge, the difficulty of orienting users in their surroundings, and the inability to estimate distances from users' camera feeds are key challenges to sighted agents. To address these challenges, researchers have suggested assisting agents with computer vision technologies, especially 3D reconstruction. This paper presents a high-fidelity prototype of such an RSA, where agents use interactive 3D maps with localization capability. We conducted a walkthrough study with thirteen agents and one user with simulated vision impairment using this prototype. The study revealed that, compared to baseline RSA, the agents were significantly faster in providing navigational assistance to users, and their mental workload was significantly reduced -- all indicate the feasibility and prospect of 3D maps in RSA.