RePFormer: Refinement Pyramid Transformer for Robust Facial Landmark Detection
This work addresses the challenge of handling complex real-world scenarios in facial landmark detection, which is important for applications like computer vision and biometrics, but it appears incremental as it builds on existing transformer and refinement methods.
The paper tackles the problem of robust facial landmark detection by introducing RePFormer, which refines landmark queries using pyramid memories and dynamic refinement, achieving superior performance on four benchmarks.
This paper presents a Refinement Pyramid Transformer (RePFormer) for robust facial landmark detection. Most facial landmark detectors focus on learning representative image features. However, these CNN-based feature representations are not robust enough to handle complex real-world scenarios due to ignoring the internal structure of landmarks, as well as the relations between landmarks and context. In this work, we formulate the facial landmark detection task as refining landmark queries along pyramid memories. Specifically, a pyramid transformer head (PTH) is introduced to build both homologous relations among landmarks and heterologous relations between landmarks and cross-scale contexts. Besides, a dynamic landmark refinement (DLR) module is designed to decompose the landmark regression into an end-to-end refinement procedure, where the dynamically aggregated queries are transformed to residual coordinates predictions. Extensive experimental results on four facial landmark detection benchmarks and their various subsets demonstrate the superior performance and high robustness of our framework.