Facial 3D Model Registration Under Occlusions With SensiblePoints-based Reinforced Hypothesis Refinement
This addresses the challenge of facial model registration under occlusion for computer vision applications, but it appears incremental as it builds on existing methods with specific enhancements.
The paper tackles the problem of registering a 3D facial model to a 2D image under occlusion by proposing a method that synthesizes additional points (SensiblePoints) to create pose hypotheses and uses a reward function to refine and select the best hypothesis, with experimentation showing promising results.
Registering a 3D facial model to a 2D image under occlusion is difficult. First, not all of the detected facial landmarks are accurate under occlusions. Second, the number of reliable landmarks may not be enough to constrain the problem. We propose a method to synthesize additional points (SensiblePoints) to create pose hypotheses. The visual clues extracted from the fiducial points, non-fiducial points, and facial contour are jointly employed to verify the hypotheses. We define a reward function to measure whether the projected dense 3D model is well-aligned with the confidence maps generated by two fully convolutional networks, and use the function to train recurrent policy networks to move the SensiblePoints. The same reward function is employed in testing to select the best hypothesis from a candidate pool of hypotheses. Experimentation demonstrates that the proposed approach is very promising in solving the facial model registration problem under occlusion.