Uncertainty Calibration for Deep Audio Classifiers
This work addresses uncertainty calibration for audio classification, which is important for improving reliability in applications like sound recognition, but it is incremental as it empirically evaluates existing methods on new data.
The paper tackled the problem of uncertainty calibration for deep audio classifiers, finding that uncalibrated models are often over-confident and that the spectral-normalized Gaussian process (SNGP) method performed best and efficiently on environment sound and music genre classification datasets.
Although deep Neural Networks (DNNs) have achieved tremendous success in audio classification tasks, their uncertainty calibration are still under-explored. A well-calibrated model should be accurate when it is certain about its prediction and indicate high uncertainty when it is likely to be inaccurate. In this work, we investigate the uncertainty calibration for deep audio classifiers. In particular, we empirically study the performance of popular calibration methods: (i) Monte Carlo Dropout, (ii) ensemble, (iii) focal loss, and (iv) spectral-normalized Gaussian process (SNGP), on audio classification datasets. To this end, we evaluate (i-iv) for the tasks of environment sound and music genre classification. Results indicate that uncalibrated deep audio classifiers may be over-confident, and SNGP performs the best and is very efficient on the two datasets of this paper.