What and When to Learn: CURriculum Ranking Loss for Large-Scale Speaker Verification
This work addresses robust speaker verification for large-scale, imperfect data, representing a new paradigm rather than an incremental improvement.
The paper tackles the challenge of speaker verification at large scale by proposing CURriculum Ranking (Curry), an adaptive loss that estimates sample difficulty online to address noisy gradients from mislabeled or degraded samples, resulting in reductions of EER by 86.8% on VoxCeleb1-O and 60.0% on SITW compared to a baseline.
Speaker verification at large scale remains an open challenge as fixed-margin losses treat all samples equally regardless of quality. We hypothesize that mislabeled or degraded samples introduce noisy gradients that disrupt compact speaker manifolds. We propose Curry (CURriculum Ranking), an adaptive loss that estimates sample difficulty online via Sub-center ArcFace: confidence scores from dominant sub-center cosine similarity rank samples into easy, medium, and hard tiers using running batch statistics, without auxiliary annotations. Learnable weights guide the model from stable identity foundations through manifold refinement to boundary sharpening. To our knowledge, this is the largest-scale speaker verification system trained to date. Evaluated on VoxCeleb1-O, and SITW, Curry reduces EER by 86.8\% and 60.0\% over the Sub-center ArcFace baseline, establishing a new paradigm for robust speaker verification on imperfect large-scale data.