Crowdsourced Multilingual Speech Intelligibility Testing
This work addresses the need for scalable and multilingual intelligibility testing in audio generation, but it is incremental as it builds on existing crowdsourcing concepts without a major breakthrough.
The paper tackles the problem of evaluating speech intelligibility for generative audio features by proposing a crowdsourced assessment approach, resulting in the public release of multilingual speech data and early experimental results.
With the advent of generative audio features, there is an increasing need for rapid evaluation of their impact on speech intelligibility. Beyond the existing laboratory measures, which are expensive and do not scale well, there has been comparatively little work on crowdsourced assessment of intelligibility. Standards and recommendations are yet to be defined, and publicly available multilingual test materials are lacking. In response to this challenge, we propose an approach for a crowdsourced intelligibility assessment. We detail the test design, the collection and public release of the multilingual speech data, and the results of our early experiments.