Mobile Sound Recognition for the Deaf and Hard of Hearing
This addresses situational awareness for deaf and hard of hearing individuals, but it is incremental as it builds on existing sound recognition methods for a specific assistive application.
The study tackled the problem of situational awareness for deaf and hard of hearing people by developing a mobile environmental sound recognition system that processes audio on-device and provides visual feedback with confidence levels, with user testing yielding important feedback.
Human perception of surrounding events is strongly dependent on audio cues. Thus, acoustic insulation can seriously impact situational awareness. We present an exploratory study in the domain of assistive computing, eliciting requirements and presenting solutions to problems found in the development of an environmental sound recognition system, which aims to assist deaf and hard of hearing people in the perception of sounds. To take advantage of smartphones computational ubiquity, we propose a system that executes all processing on the device itself, from audio features extraction to recognition and visual presentation of results. Our application also presents the confidence level of the classification to the user. A test of the system conducted with deaf users provided important and inspiring feedback from participants.