Evaluation Framework for Computer Vision-Based Guidance of the Visually Impaired
This work addresses the problem of inconsistent evaluation in assistive technology for visually impaired individuals, though it is incremental as it builds on existing guidance systems.
The paper tackles the lack of a standardized evaluation methodology for computer vision-based guidance systems for visually impaired persons by proposing a framework that compares computer vision decisions to human-labeled reference decisions for critical situations like crosswalks and stairs.
Visually impaired persons have significant problems in their everyday movement. Therefore, some of our previous work involves computer vision in developing assistance systems for guiding the visually impaired in critical situations. Some of those situations includes crosswalks on road crossings and stairs in indoor and outdoor environment. This paper presents an evaluation framework for computer vision-based guiding of the visually impaired persons in such critical situations. Presented framework includes the interface for labeling and storing referent human decisions for guiding directions and compares them to computer vision-based decisions. Since strict evaluation methodology in this research field is not clearly defined and due to the specifics of the transfer of information to visually impaired persons, evaluation criterion for specific simplified guiding instructions is proposed.