Estimating the confidence of speech spoofing countermeasure
This work addresses the reliability of spoofing detection in speech systems, though it is incremental as it builds on existing countermeasure methods.
The paper tackled the problem of estimating confidence for speech spoofing countermeasures to handle unknown attacks, finding that energy-based and neural-network-based estimators achieved acceptable performance on the ASVspoof2019 database but faced challenges with additional unknown data.
Conventional speech spoofing countermeasures (CMs) are designed to make a binary decision on an input trial. However, a CM trained on a closed-set database is theoretically not guaranteed to perform well on unknown spoofing attacks. In some scenarios, an alternative strategy is to let the CM defer a decision when it is not confident. The question is then how to estimate a CM's confidence regarding an input trial. We investigated a few confidence estimators that can be easily plugged into a CM. On the ASVspoof2019 logical access database, the results demonstrate that an energy-based estimator and a neural-network-based one achieved acceptable performance in identifying unknown attacks in the test set. On a test set with additional unknown attacks and bona fide trials from other databases, the confidence estimators performed moderately well, and the CMs better discriminated bona fide and spoofed trials that had a high confidence score. Additional results also revealed the difficulty in enhancing a confidence estimator by adding unknown attacks to the training set.