An End-to-End Neural Network for Image-to-Audio Transformation
This work addresses accessibility for vision-impaired or distracted users on low-resource devices, but it is incremental as it builds on existing image-to-text and text-to-speech methods.
The paper tackles the problem of accessibility for vision-impaired users by developing an end-to-end neural network for image-to-audio transformation, resulting in a system that is 29% faster and uses 19% fewer parameters with a 2% reduction in phone accuracy compared to non-end-to-end approaches.
This paper describes an end-to-end (E2E) neural architecture for the audio rendering of small portions of display content on low resource personal computing devices. It is intended to address the problem of accessibility for vision-impaired or vision-distracted users at the hardware level. Neural image-to-text (ITT) and text-to-speech (TTS) approaches are reviewed and a new technique is introduced to efficiently integrate them in a way that is both efficient and back-propagate-able, leading to a non-autoregressive E2E image-to-speech (ITS) neural network that is efficient and trainable. Experimental results are presented showing that, compared with the non-E2E approach, the proposed E2E system is 29% faster and uses 19% fewer parameters with a 2% reduction in phone accuracy. A future direction to address accuracy is presented.