Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition
This work addresses performance degradation in face recognition for low resolution images, which is an incremental improvement for applications like surveillance or mobile devices.
The paper tackles the problem of low resolution face recognition by proposing an attention similarity knowledge distillation method that transfers attention maps from a high resolution teacher network to a low resolution student network, improving recognition performance and outperforming state-of-the-art results on various benchmarks.
Deep learning has achieved outstanding performance for face recognition benchmarks, but performance reduces significantly for low resolution (LR) images. We propose an attention similarity knowledge distillation approach, which transfers attention maps obtained from a high resolution (HR) network as a teacher into an LR network as a student to boost LR recognition performance. Inspired by humans being able to approximate an object's region from an LR image based on prior knowledge obtained from HR images, we designed the knowledge distillation loss using the cosine similarity to make the student network's attention resemble the teacher network's attention. Experiments on various LR face related benchmarks confirmed the proposed method generally improved recognition performances on LR settings, outperforming state-of-the-art results by simply transferring well-constructed attention maps. The code and pretrained models are publicly available in the https://github.com/gist-ailab/teaching-where-to-look.