ICASSP 2021 Acoustic Echo Cancellation Challenge: Datasets, Testing Framework, and Results
This challenge aims to improve AEC for audio communication and conferencing systems by providing better datasets and testing, but it is incremental as it builds on existing research without introducing new methods.
The ICASSP 2021 Acoustic Echo Cancellation Challenge addressed the problem of acoustic echo cancellation (AEC) performance degrading on real recordings and poor correlation of conventional metrics with subjective quality in noisy environments, by open-sourcing large real and synthetic datasets and a test framework, with winners selected based on average Mean Opinion Score (MOS).
The ICASSP 2021 Acoustic Echo Cancellation Challenge is intended to stimulate research in the area of acoustic echo cancellation (AEC), which is an important part of speech enhancement and still a top issue in audio communication and conferencing systems. Many recent AEC studies report good performance on synthetic datasets where the train and test samples come from the same underlying distribution. However, the AEC performance often degrades significantly on real recordings. Also, most of the conventional objective metrics such as echo return loss enhancement (ERLE) and perceptual evaluation of speech quality (PESQ) do not correlate well with subjective speech quality tests in the presence of background noise and reverberation found in realistic environments. In this challenge, we open source two large datasets to train AEC models under both single talk and double talk scenarios. These datasets consist of recordings from more than 2,500 real audio devices and human speakers in real environments, as well as a synthetic dataset. We open source two large test sets, and we open source an online subjective test framework for researchers to quickly test their results. The winners of this challenge will be selected based on the average Mean Opinion Score (MOS) achieved across all different single talk and double talk scenarios.