Holistic Uncertainty Estimation For Open-Set Recognition
This work addresses uncertainty estimation for open-set recognition in biometrics, which is an incremental improvement over existing methods.
The paper tackles the problem of accurate uncertainty estimation in open-set recognition by proposing HolUE, a holistic method that accounts for both gallery uncertainty and embedding uncertainty, and demonstrates its effectiveness in identifying recognition errors across multiple datasets including IJB-C, VoxBlink, and a new whale and dolphin protocol.
Accurate uncertainty estimation is a critical challenge in open-set recognition, where a probe biometric sample may belong to an unknown identity. It can be addressed through sample quality estimation via probabilistic embeddings. However, the low variance of probabilistic embedding only partly implies a low identification error probability: an embedding of a sample could be close to several classes in a gallery, thus yielding high uncertainty despite high sample quality. We propose HolUE - a holistic uncertainty estimation method based on a Bayesian probabilistic model; it is aware of two sources of ambiguity in the open-set recognition system: (1) the gallery uncertainty caused by overlapping classes and (2) the uncertainty of embeddings. Challenging open-set recognition datasets, such as IJB-C for the image domain and VoxBlink for the audio domain, serve as a testbed for our method. We also provide a new open-set recognition protocol for the identification of whales and dolphins. In all cases, HolUE better identifies recognition errors than alternative uncertainty estimation methods, including those based solely on sample quality.