OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibility
This addresses occlusion challenges in facial landmark detection for applications involving human-like faces, including humans and stylized characters, but is incremental as it builds on existing heatmap-based methods.
The paper tackles the problem of facial landmark detection under occlusion by proposing OccFace, a framework that jointly predicts landmark coordinates and per-point visibility, resulting in improved robustness on occluded regions while maintaining accuracy on visible landmarks, with metrics like NME, Occ AP, F1@0.5, and ROC-AUC reported.
Accurate facial landmark detection under occlusion remains challenging, especially for human-like faces with large appearance variation and rotation-driven self-occlusion. Existing detectors typically localize landmarks while handling occlusion implicitly, without predicting per-point visibility that downstream applications can benefits. We present OccFace, an occlusion-aware framework for universal human-like faces, including humans, stylized characters, and other non-human designs. OccFace adopts a unified dense 100-point layout and a heatmap-based backbone, and adds an occlusion module that jointly predicts landmark coordinates and per-point visibility by combining local evidence with cross-landmark context. Visibility supervision mixes manual labels with landmark-aware masking that derives pseudo visibility from mask-heatmap overlap. We also create an occlusion-aware evaluation suite reporting NME on visible vs. occluded landmarks and benchmarking visibility with Occ AP, F1@0.5, and ROC-AUC, together with a dataset annotated with 100-point landmarks and per-point visibility. Experiments show improved robustness under external occlusion and large head rotations, especially on occluded regions, while preserving accuracy on visible landmarks.