CoverTheFace: face covering monitoring and demonstrating using deep learning and statistical shape analysis
This work addresses the practical issue of mask-wearing compliance for public health, though it is incremental as it builds on existing detection methods by adding a demonstration component.
The paper tackled the problem of improper mask-wearing during the COVID-19 pandemic by developing an automated system that not only detects face coverings but also generates personalized visual demonstrations for correct usage, achieving noticeable improvements in generating examples, especially for half-profile face images.
Wearing a mask is a strong protection against the COVID-19 pandemic, even though the vaccine has been successfully developed and is widely available. However, many people wear them incorrectly. This observation prompts us to devise an automated approach to monitor the condition of people wearing masks. Unlike previous studies, our work goes beyond mask detection; it focuses on generating a personalized demonstration on proper mask-wearing, which helps people use masks better through visual demonstration rather than text explanation. The pipeline starts from the detection of face covering. For images where faces are improperly covered, our mask overlay module incorporates statistical shape analysis (SSA) and dense landmark alignment to approximate the geometry of a face and generates corresponding face-covering examples. Our results show that the proposed system successfully identifies images with faces covered properly. Our ablation study on mask overlay suggests that the SSA model helps to address variations in face shapes, orientations, and scales. The final face-covering examples, especially half profile face images, surpass previous arts by a noticeable margin.