Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata
This work addresses speech intelligibility assessment for hearing aid development, representing an incremental improvement over existing methods.
The paper tackled the problem of automated speech intelligibility prediction for hearing aids by introducing MBI-Net+, which improved accuracy by incorporating Whisper embeddings, speech metadata, and the HASPI metric, surpassing baseline systems on the Clarity Prediction Challenge 2023 dataset.
Automated speech intelligibility assessment is pivotal for hearing aid (HA) development. In this paper, we present three novel methods to improve intelligibility prediction accuracy and introduce MBI-Net+, an enhanced version of MBI-Net, the top-performing system in the 1st Clarity Prediction Challenge. MBI-Net+ leverages Whisper's embeddings to create cross-domain acoustic features and includes metadata from speech signals by using a classifier that distinguishes different enhancement methods. Furthermore, MBI-Net+ integrates the hearing-aid speech perception index (HASPI) as a supplementary metric into the objective function to further boost prediction performance. Experimental results demonstrate that MBI-Net+ surpasses several intrusive baseline systems and MBI-Net on the Clarity Prediction Challenge 2023 dataset, validating the effectiveness of incorporating Whisper embeddings, speech metadata, and related complementary metrics to improve prediction performance for HA.