AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations
This addresses the need for more stable and efficient training in face recognition systems, though it is incremental as it builds on existing cosine-based softmax losses.
The paper tackles the problem of hyperparameter sensitivity in cosine-based softmax losses for face recognition by proposing AdaCos, a hyperparameter-free loss that adaptively scales cosine logits, achieving state-of-the-art accuracy on LFW, MegaFace, and IJB-C datasets.
The cosine-based softmax losses and their variants achieve great success in deep learning based face recognition. However, hyperparameter settings in these losses have significant influences on the optimization path as well as the final recognition performance. Manually tuning those hyperparameters heavily relies on user experience and requires many training tricks. In this paper, we investigate in depth the effects of two important hyperparameters of cosine-based softmax losses, the scale parameter and angular margin parameter, by analyzing how they modulate the predicted classification probability. Based on these analysis, we propose a novel cosine-based softmax loss, AdaCos, which is hyperparameter-free and leverages an adaptive scale parameter to automatically strengthen the training supervisions during the training process. We apply the proposed AdaCos loss to large-scale face verification and identification datasets, including LFW, MegaFace, and IJB-C 1:1 Verification. Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy. Our method outperforms state-of-the-art softmax losses on all the three datasets.