Semantic and structural image segmentation for prosthetic vision
This work addresses the challenge of scene understanding for visually impaired people using prosthetic vision, but it is incremental as it builds on existing computer vision techniques for a specific application.
The paper tackled the problem of limited information in prosthetic vision by developing a method that combines convolutional neural networks to extract structural edges and object silhouettes for indoor scenes, resulting in good accuracy for object recognition and room identification tasks compared to other methods.
Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the simulated prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.