AttFC: Attention Fully-Connected Layer for Large-Scale Face Recognition with One GPU
This addresses memory and time constraints for researchers and practitioners training face recognition models on large datasets, though it is incremental as it builds on existing FC layer improvements.
The paper tackles the problem of high computational resource demands in large-scale face recognition by proposing the AttFC layer, which reduces model parameters and enables training on one GPU with comparable performance to state-of-the-art methods.
Nowadays, with the advancement of deep neural networks (DNNs) and the availability of large-scale datasets, the face recognition (FR) model has achieved exceptional performance. However, since the parameter magnitude of the fully connected (FC) layer directly depends on the number of identities in the dataset. If training the FR model on large-scale datasets, the size of the model parameter will be excessively huge, leading to substantial demand for computational resources, such as time and memory. This paper proposes the attention fully connected (AttFC) layer, which could significantly reduce computational resources. AttFC employs an attention loader to generate the generative class center (GCC), and dynamically store the class center with Dynamic Class Container (DCC). DCC only stores a small subset of all class centers in FC, thus its parameter count is substantially less than the FC layer. Also, training face recognition models on large-scale datasets with one GPU often encounter out-of-memory (OOM) issues. AttFC overcomes this and achieves comparable performance to state-of-the-art methods.